Graphcore IPUs for GNNs are freely available on Paperspace

IPUs (Intelligence Processing Unit) by UK-based Graphcore is a new type of hardware (chips and servers) tailored for AI compute - including optimized sparse matrix multiplications. Sparse operations are the main building block of GNNs but are still one of the slowest operations on GPUs (tailored for dense matrix multiplications).

The ImageNet moment in 2012 happened thanks to the hardware lottery as well - when we found that GPUs are dramatically better than CPUs in training deep nets. IPUs can well be the winning hardware lottery ticket for GNNs!

In the recent blog post, Michael Bronstein, Emanuele Rossi, and Daniel Justus hinted upon spectacular performance gains when training Temporal Graph Nets (TGN): 3-11x faster on a single IPU chip compared to A100. IPUs also deliver great general performance in MLPerf, the biggest go-to benchmark of efficient training large vision and language models.

Today, you can try running the code for free on IPU-POD16 that has four IPU chips thanks to the partnership between Paperspace and Graphcore. In addition to standard BERT, RoBERTa and ViTs, Graphcore prepared modules with Cluster-GCN, TGN, and SchNet (a popular baseline for molecular dynamics).

You can run most of PyTorch / TensorFlow code, and IPUs should natively support XLA, so it’s a good time to catch up with JAX and its GNN libraries like Jraph 😉