DeeperGCN All You Need to Train Deeper GCNs

目录

• 概
• 符号说明
• 广义的 aggregation function
• 代码

Li G., Xiong C., Thabet A. and Ghanem B. DeeperGcn: all you need to train deeper gcns. arXiv preprint arXiv:2006.07739

概

本文介绍了一种连续可微的 aggregation function (极限状态下能够 cover 常用的 mean, max).

符号说明

• \(\mathcal{G} = (\mathcal{V}, \mathcal{E})\), 图;
• \(\mathbf{h}_v \in \mathbb{R}^D\), node features;
• \(\mathbf{h}_e \in \mathbb{R}^C\), edge features;
• \(\mathcal{N}(v)\), node \(v\) 的一阶邻居.

广义的 aggregation function

• 一般的 GCN 每一层需要历经:

  \[\mathbf{m}_{vu}^{(l)} = \bm{\rho}^{(l)} (\mathbf{h}_v^{(l)}, \mathbf{h}_v^{(l)}, \mathbf{h}_{e_{vu}}^{(l)}), \: u \in \mathcal{N}(v), \\ \mathbf{m}_v^{(l)} = \bm{\zeta}^{(l)}(\{\mathbf{m}_{vu}^{(l)}| u \in \mathcal{N}(v)\}), \\ \mathbf{h}_v^{(l+1)} = \bm{\phi}^{(l)} (\mathbf{h}_v^{(l)}, \mathbf{m}_v^{(l)}). \]

  其中 \(\bm{\rho}, \bm{\zeta}, \bm{\phi}\) 为可训练的函数. 这里我们主要关注 \(\bm{\zeta}\).

• SoftMax Aggregation:

  \[\bm{\zeta}_{\beta}(\{\mathbf{m}_{vu}^{(l)}| u \in \mathcal{N}(v)\}) = \sum_{u \in \mathcal{N}_v} \frac{\exp(\beta \mathbf{m}_{vu})}{\sum_{i \in \mathcal{N}(v)} \exp(\beta \mathbf{m}_{vi})} \cdot \mathbf{m}_{vu}. \]

  显然当 \(\beta \rightarrow 0\) 的时候它退化为 \(\text{Mean}(\cdot)\), 当 \(\beta \rightarrow +\infty\) 的时候退化为 \(\text{Max}(\cdot)\).

• PowerMean Aggregation:

  \[\bm{\zeta}_{p}(\{\mathbf{m}_{vu}^{(l)}| u \in \mathcal{N}(v)\}) =(\frac{1}{|\mathcal{N}_v|} \sum_{u \in \mathcal{N}_v} \mathbf{m}_{vu}^p)^{1/p}, \]

  当 \(p = 1\) 的时候, 退化为 \(\text{Mean}(\cdot)\), 当 \(p \rightarrow +\infty\) 的时候退化为 \(\text{Max}(\cdot)\), 当 \(p=-1, p \rightarrow 0\) 的时候分别退化为 harmonic 和 geometric mean aggregation.

代码

official