Learning Graph Structure With A FiniteState Automaton Layer
Abstract
Graphbased neural network models are producing strong results in a number of domains, in part because graphs provide flexibility to encode domain knowledge in the form of relational structure (edges) between nodes in the graph. In practice, edges are used both to represent intrinsic structure (e.g., abstract syntax trees of programs) and more abstract relations that aid reasoning for a downstream task (e.g., results of relevant program analyses). In this work, we study the problem of learning to derive abstract relations from the intrinsic graph structure. Motivated by their power in program analyses, we consider relations defined by paths on the base graph accepted by a finitestate automaton. We show how to learn these relations endtoend by relaxing the problem into learning finitestate automata policies on a graphbased POMDP and then training these policies using implicit differentiation. The result is a differentiable Graph FiniteState Automaton (GFSA) layer that adds a new edge type (expressed as a weighted adjacency matrix) to a base graph. We demonstrate that this layer can find shortcuts in gridworld graphs and reproduce simple static analyses on Python programs. Additionally, we combine the GFSA layer with a larger graphbased model trained endtoend on the variable misuse program understanding task, and find that using the GFSA layer leads to better performance than using handengineered semantic edges or other baseline methods for adding learned edge types.
 Publication:

arXiv eprints
 Pub Date:
 July 2020
 arXiv:
 arXiv:2007.04929
 Bibcode:
 2020arXiv200704929J
 Keywords:

 Computer Science  Machine Learning;
 Statistics  Machine Learning
 EPrint:
 Submitted to NeurIPS 2020