Effortless Distributed Training of Ultra-Wide GCNs

A great post about distributed training of GNNs on large graphs. The architecture splits the GNN into several submodules where each is trained independently on separate GPUs, providing the flexibility to increase significantly the hidden dimension of embeddings. As such this approach is GCN model agnostic, compatible with existing sampling methods, and performs the best in very large graphs.