Optimizing Graph Neural Network Training on Large Graphs
Name
Stathas-nistath-meng-eecs-2021-thesis.pdf
Description
Thesis PDF
Size
1.09 MB
Format
Adobe PDF
Checksum (MD5)
1a037d34485b817b75c780b0268e12b7
Author(s)
Stathas, Nickolas
Advisor(s)
Leiserson, Charles E.
Kaler, Tim
Date Issued
June 2021
Publisher
Massachusetts Institute of Technology
Abstract
Graphs can be used to represent many important classes of structured real-world data. For this reason, there has been an increase of research interest in various machine learning approaches to solve tasks such as link prediction and node property prediction. Graph Neural Network models demonstrate good performance on such tasks. However, the depth of the models and the size of the graphs they can be trained on is constrained either by the low processing throughput of CPUs or by the limited memory capacity of GPUs. Techniques such as neighborhood sampling are often used to create smaller mini-batch training examples that fit in GPU memory. In this thesis, I provide a systematic performance analysis of GNN training codes written using PyTorch Geometric, the most popular machine learning framework for GNNs. Through this performance analysis, I uncover significant performance bottlenecks related to neighborhood sampling and GPU data transfers. To address these issues, I create FastPyG: a performance-engineered fork of PyTorch Geometric, which achieves a 3-6× speedup over comparable PyTorch Geometric codes without impacting model accuracy. The core contribution included in FastPyG is fast_sampler, an efficient and parallel neighborhood sampling implementation in C++.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright MIT
Persistent DSpace Link
