Adaptive Message Passing: A General Framework to Mitigate Oversmoothing, Oversquashing, and Underreaching

Guest post by Federico Errica

📖 Blog post: link
⚗️ Paper: http://arxiv.org/abs/2312.16560

Long-range interactions are essential for the correct description of complex systems in many scientific fields. Recently, deep graph networks have been employed as efficient, data-driven surrogate models for predicting properties of complex systems represented as graphs. In practice, most deep graph networks cannot really model long-range dependencies due to the intrinsic limitations of (synchronous) message passing, namely oversmoothing, oversquashing, and underreaching.

Motivated by these observations, we propose Adaptive Message Passing (AMP) to let the DGN decide how many messages each node should send -up to infinity! - and when to send them. In other words:

1️⃣ We learn the depth of the network during training (addressing underreaching)

2️⃣ We apply a differentiable, soft filter on messages sent by nodes, which in principle can completely shut down the propagation of a message (addressing oversmoothing and oversquashing).

❗️ AMP can easily and automatically improve the performances of your favorite message passing architecture, e.g., GCN/GIN. ❗️

We believe AMP will foster exciting research opportunities in the graph machine learning field and find successful applications in the fields of physics, chemistry, and material sciences.