Graph Machine Learning

Modeling Intelligence via Graph Neural Networks: slides

The slides of the thesis by Keyulu Xu: Modeling Intelligence via Graph Neural Networks. Keyulu is one of the authors of GIN and other notable works in GML.

Stanford Graph Learning Workshop

A great online workshop will be organized by Stanford, on Thursday, Sept 16 2021, 08:00 - 17:00 Pacific Time. It includes talks from Jure Leskovec, Matthias Fey, Weihua Hu, Jiaxuan You, as well as a series of talks on applications of GNNs, and two industry panels.

Fresh picks from ArXiv
This week on ArXiv: GNN link with causal models, augmenting data, and using knowledge graphs with BERT 🧸

If I forgot to mention your paper, please shoot me a message and I will update the post.

GNNs
* Relating Graph Neural Networks to Structural Causal Models with Petar Veličković, Kristian Kersting
* A Study of Joint Graph Inference and Forecasting with Stephan Günnemann
* Is Heterophily A Real Nightmare For Graph Neural Networks To Do Node Classification?
* Local Augmentation for Graph Neural Networks
* Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERT EMNLP 2021

Math
* There does not exist a strongly regular graph with parameters (1911,270,105,27)
* On the Fundamental Limits of Matrix Completion: Leveraging Hierarchical Similarity Graphs

PyG 2.0 (PyTorch Geometric 2.0) Release

One of the most prominent libraries in the world of GNNs and Geometric DL got a major update (and a small re-branding to a shorter "PyG")! Now with a website and Slack channel.

In addition to a constantly growing number of supported GNN architectures, the 2.0 version features:

1. Heterogeneous graph support with models, mini-batching, sampling, and a one-line conversion of homogeneous models to heterogeneous.

2. GraphGym - a whole platform for designing and experimenting with GNN architectures where you can fine-tune the nitty-gritty details of your model and find the best hyperparams. Based on the NeurIPS'20 paper

3. Pre-defined models - before, you'd usually build a GNN model from a collection of layers by yourself (trying to not forget to put that non-linearity after the GCN layer). Now, the library includes 25 well-known models!

4. Half-precision support and other smaller improvements to make your GNN journey easier.

Organizational Update

I've been running this channel alone for almost two years but it's been more challenging recently to keep the previous pace. To help me, Michael Galkin generously accepted to be one of the admins of this channel, who has been already involved in several posts here.

Michael Galkin is a postdoc at Mila & McGill and you can know him by the amazing digests of knowledge graphs papers, contributions to the open-source projects, and strong research works. Please, welcome Michael and subscribe to his twitter.

P.S. Also I will use this opportunity to remind that if you have something to share with a graph community, do not hesitate to contact us.

Review: Deep Learning on Sets

A new blog post by Fabian Fuchs and others about recent approaches of applying deep learning on sets. It digests several paradigms such as permuting & averaging, sorting, approximating invariance, and learning on graphs as a way to overcome permutation invariance of machine learning algorithms.

Graph ML in Industry Workshop

When I wrote top applications of GNNs at the beginning of this year, I had a feeling that graph ML community is mature enough to start being used in industrial companies. Nine months ahead we decided to gather researchers, engineers, and industry professionals to talk about applications of graphs in the companies. Please, join us on 23rd Sept, 17-00 Paris time (free, online, ~3 hours) by registering at the link.

Researcher Positions at Dimitri's Ognibene's Lab

Two positions for post doc/researchers are available at Milano Bicocca University under Dimitri's Ognibene supervision. 2 years contract, based in Milan (possibility to remote working). For application contact: Dimitri Ognibene dimitri.ognibene@unimib.it. Description is below:

Do social media harm teenagers and our society?
Can we make them safer?

We will use the state of the art in graph neural networks, reinforcement learning, nlp, cv, and machine learning in general to improve our understanding of social media dynamics, and help our society by supporting and teaching young people tackle hate speech and fake news in social media.

The Learning on Graphs and Geometry Reading Group

A reading group organized by Hannes Stärk with supervision from Pietro Liò at Cambridge. Includes really interesting fresh papers on graphs. Every Tuesday at 5pm CEST.

Fresh picks from ArXiv
This week on ArXiv: improving robustness by resampling a graph, learning better scenes, and new homophily definitions 🐤

If I forgot to mention your paper, please shoot me a message and I will update the post.

GNNs
* Training Graph Neural Networks by Graphon Estimation
* Learning to Generate Scene Graph from Natural Language Supervision ICCV 2021
* Pointspectrum: Equivariance Meets Laplacian Filtering for Graph Representation Learning
* Sparsifying the Update Step in Graph Neural Networks
* Adaptive Label Smoothing To Regularize Large-Scale Graph Training

Math
* How likely is a random graph shift-enabled?
* The Popularity-Homophily Index: A new way to measure Homophily in Directed Graphs

Monday Theory: Structural vs Positional Node Representations

In the new slide deck, Bruno Ribeiro (Purdue University) uncovers the nature of two commonly used mechanisms for building node representations. Structural representations are permutation insensitive (like GNNs) whereas positional representations are permutation sensitive (like SVD vectors). Hence, all GRL approaches can be broadly classified into those two families. Takeaway messages:

Message 1: Positional representations of k nodes are to most expressive k-node structural representations as samples of a distribution are to sufficient statistics of the distribution. This is based on the results published in the ICLR'20 paper

Message 2: As soon as you introduce some sort of node IDs you break equivariance but at the same time can predict properties of any subset of nodes (better link prediction). You’d better aggregate over multiple samples though (from the stats analogy). If you stick to equivariance, you can predict node or graph-level properties but nothing in-between.

GNN Tutorial & Graph Convolution Intuition @ Distill

Distill.pub is a great new resource aimed at re-defining a way we publish papers. Publications on Distill have rich visualizations and hands-on examples that you can tweak right in a browser. Unfortunately, Distill goes on a hiatus.
But, as the last bow, the authors prepared two very cool articles breaking down message passing and graph convolutions:

1. A Gentle Introduction to Graph Neural Networks


2. Understanding Convolutions on Graphs

Something you definitely do not want to miss in September!

Fresh picks from ArXiv
This week on ArXiv: predictions of routing times, benchmarking architecture tricks, and drug repurposing study 💊

If I forgot to mention your paper, please shoot me a message and I will update the post.

Conferences
* Single Node Injection Attack against Graph Neural Networks CIKM 2021
* ETA Prediction with Graph Neural Networks in Google Maps CIKM 2021, with Petar Veličković
* Tree Decomposed Graph Neural Network CIKM 2021, with Tyler Derr
* DSKReG: Differentiable Sampling on Knowledge Graph for Recommendation with Relational GNN CIKM 2021
* Multiplex Graph Neural Network for Extractive Text Summarization EMNLP 2021
* Demystifying Drug Repurposing Domain Comprehension with Knowledge Graph Embedding IEEE BioCAS 2021
* DC-GNet: Deep Mesh Relation Capturing Graph Convolution Network for 3D Human Shape Reconstruction ACM MM'21
* Visualizing JIT Compiler Graphs GD 2021

GNNs
* Spatio-Temporal Graph Contrastive Learning

Benchmark
* Weisfeiler-Leman in the BAMBOO: Novel AMR Graph Metrics and a Benchmark for AMR Graph Similarity TACL 2021
* Bag of Tricks for Training Deeper Graph Neural Networks: A Comprehensive Benchmark Study

Math
* Smallest graphs with given automorphism group

Graph Drawing and Network Visualization 2021

A symposium on graph Drawing and network visualization is a nice niche conference on how to draw graphs efficiently and insightfully. This year it will be organized both online and offline (in Tübingen, Germany). Dates are: September 14-17, 2021. Accepted papers can be seen here.

Video: Graph Neural Networks - a perspective from the ground up

A beautiful video about GNNs aimed at CS undergrads that explains what message passing and node embeddings are and gives a link prediction example.

Fresh picks from ArXiv
This week on ArXiv: GNNs for lidars, complex reasoning in knowledge graphs, and building AI for drawing graphs 🤖

If I forgot to mention your paper, please shoot me a message and I will update the post.

GNNs
* Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection
* TabGNN: Multiplex Graph Neural Network for Tabular Data Prediction
* Learning to Match Features with Seeded Graph Matching Network
* EqGNN: Equalized Node Opportunity in Graphs


Applications
* GP-S3Net: Graph-based Panoptic Sparse Semantic Segmentation Network
* Adaptive Graph Convolution for Point Cloud Analysis
* Hyperbolic Hypergraphs for Sequential Recommendation
* Implementation of Sprouts: a graph drawing game

Knowledge graphs
* Fact-Tree Reasoning for N-ary Question Answering over Knowledge Graphs
* UNIQORN: Unified Question Answering over RDF Knowledge Graphs and Natural Language Text

TorchDrug: a powerful and flexible machine learning platform for drug discovery

Jian Tang and his co-workers from MILA open-sourced a new library TorchDrug on drug modeling with machine learning. It includes an easy interface for property prediction, pretrained molecular representations, de-novo molecule design & optimization, knowledge graph reasoning, and more.

Graph Machine Learning research groups: Ian Davidson

I do a series of posts on the groups in graph research, previous post is here. The 33rd is Ian Davidson, a professor at UC Davis, who works in the areas with societal impacts such as neuroscience, intelligent tutoring systems and social networks.

Ian Davidson (~1973)
- Affiliation: UC Davis
- Education: Ph.D. at Monash University in 2000 (advisor: C.S. Wallace)
- h-index 44
- Interests: fairness, clustering, graphical models.
- Awards: best papers at KDD, SIAM, ICDM

Book: Designing and Building Enterprise Knowledge Graphs (Synthesis Lectures on Data, Semantics, and Knowledge)

A new book by Ora Lassila and Juan Sequeda that guides on designing and building knowledge graphs from enterprise relational databases in practice. It presents a principled framework centered on mapping patterns to connect relational databases with knowledge graphs, the roles within an organization responsible for the knowledge graph, and the process that combines data and people. The content of this book is applicable to knowledge graphs being built either with property graph or RDF graph technologies.

Awesome Efficient Graph Neural Networks

A new awesome repo by Chaitanya K. Joshi with the curated list of must-read papers on efficient Graph Neural Networks and scalable Graph Representation Learning for real-world applications.