GraphML News (April 5th) - Isomorphic Round, Graph Transformers at Kumo, new blogs

Got some news!

💸 Isomorphic Labs raised a generous $600M from Thrive Capital, GV, and Alphabet in the first external round. The attached press release also mentions collaborations with pharma giants Eli Lilly and Novartis - seems like whatever comes next after AlphaFold 3 looks quite appealing to the industry. We’ll keep you posted in our Geometric Wall Street Bulletin.

🏵️ Looking at LLM guts from the graph learning perspective becomes popular: Anthropic posted a massive study in two papers and lots of visual material on AI biology with strong graph vibes - LLMs perform multi-hop reasoning with concept graphs in mind, and you can actually identify circuits (DAGs) of activations doing certain kind of computation.

🚚 Kumo published a nice blog post on using graph transformers at scale in relational DL tasks. Perhaps the most insightful part is about positional encodings - as graphs are large (10M+ nodes in RelBench), global PEs don’t really scale up so they have to resort to more local options like hop encoding or relative PEs. Besides, there is a need for time encoding as the e-commerce graphs are always temporal. Experiments on RelBench bring some noticeable improvements.

⌛ Bryan Perozzi from Google Research (the OG of DeepWalk) wrote a review post looking at the milestones of graph learning from pre-historic times (pre-2013) to the most recent applications (those we often highlight in this channel).

Weekend reading:

Why do LLMs attend to the first token? by Federico Barbero, Alvaro Arroyo, and all the familiar folks from Transformers Need Glasses - here the authors study attention sinks and draw parallels to over-squashing and representational collapse.

On that note, don’t miss the talk by Petar Veličković on LLMs as Graph Neural Networks at the recent GLOW reading group - it adds much more context to the research area and hints at the above paper.

Affordable AI Assistants with Knowledge Graph of Thoughts by Maciej Besta and 13 (👀) co-authors. Maciej is the author of the famous Graph of Thoughts, here it’s extended to KGs and agentic environments.