Strategies for pre-training graph neural networks

W Hu, B Liu, J Gomes, M Zitnik, P Liang… - arXiv preprint arXiv …, 2019 - arxiv.org
… Here, we focus on pre-training as an approach to transfer learning in Graph Neural Networks
(GNNs) (Kipf & Welling, 2017; Hamilton et al., 2017a; Ying et al., 2018b; Xu et al., 2019; …

Training graph neural networks with 1000 layers

G Li, M Müller, B Ghanem… - … on machine learning, 2021 - proceedings.mlr.press
… We study deep weight-tied GNNs that have the parameter cost of only a single layer. We …
a deep graph equilibrium GNN, which is essentially a weight-tied network with infinite depth. …

Learning to pre-train graph neural networks

Y Lu, X Jiang, Y Fang, C Shi - Proceedings of the AAAI conference on …, 2021 - ojs.aaai.org
… Thus, it is crucial to devise a self-supervised strategy to pre-train graph-level representations…
, a pretraining strategy for GNNs that learns to pre-train (L2P) at both node and graph levels …

Graphnorm: A principled approach to accelerating graph neural network training

T Cai, S Luo, K Xu, D He, T Liu… - … on Machine Learning, 2021 - proceedings.mlr.press
… to help the optimization of deep neural networks. Curiously, … normalization is effective for
Graph Neural Networks (GNNs). … to the heavy batch noise in graph datasets. Second, we show …

Deep neural networks for learning graph representations

S Cao, W Lu, Q Xu - Proceedings of the AAAI conference on artificial …, 2016 - ojs.aaai.org
… • Theoretically, we argue that the deep neural networks offer the advantage of being able to
capture non-linear information conveyed by the graph, whereas such information can not be …

Accurate, efficient and scalable training of Graph Neural Networks

H Zeng, H Zhou, A Srivastava, R Kannan… - Journal of Parallel and …, 2021 - Elsevier
Graph Neural Networks (GNNs) are powerful deep learning models to generate node
embeddings on graphs. … Then in Section 4.4, we show how to extend our strategies to other graph

Deep learning on graphs: A survey

Z Zhang, P Cui, W Zhu - IEEE Transactions on Knowledge and …, 2020 - ieeexplore.ieee.org
… different types of deep learning methods on graphs. We divide … training strategies: graph
recurrent neural networks, graphgraphs, which are also referred to as graph neural networks (…

DistDGL: Distributed graph neural network training for billion-scale graphs

D Zheng, C Ma, M Wang, J Zhou, Q Su… - 2020 IEEE/ACM 10th …, 2020 - ieeexplore.ieee.org
… There are potentially two types of model parameters in graph neural networks. f, … effectiveness,
we compare DistDGL’s graph partitioning algorithm with two alternatives: random graph

A comprehensive survey on graph neural networks

Z Wu, S Pan, F Chen, G Long, C Zhang… - … on neural networks …, 2020 - ieeexplore.ieee.org
… Recently, many studies on extending deep learning approaches for graph data have …
overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose …

A comprehensive survey on distributed training of graph neural networks

H Lin, M Yan, X Ye, D Fan, S Pan… - Proceedings of the …, 2023 - ieeexplore.ieee.org
… of distributed training of graph neural networks by investigating … ABSTRACT | Graph neural
networks (GNNs) have been … for their effectiveness in learning over graphs. To scale GNN …