Discovering Symbolic Models from Deep Learning with Inductive Biases

In addition to the previous post of learning dynamics of particles, there is another NeurIPS 2020 work by Cranmer et al. of learning new equations of unknown physics. The idea is to use the messages on edges of GNNs and node outputs as the input for symbolic regression algorithm (eureuka), which does a genetic search of the input and symbols such as +, -, exp, log, etc.

What's interesting is that this methodology can be applied to any NN model, when symbolic regression is used to unravel a compact symbolic expression of the underlying data. Moreover, one can see genetic search as some post-processing of the GNN model discovering new, closed-formula solutions to the data that generalizes better than GNN itself. Blog post (with code) and Yannic explanation are available.