Augmenting Neural Networks with First-order Logic
Abstract
Today, the dominant paradigm for training neural networks involves minimizing task loss on a large dataset. Using world knowledge to inform a model, and yet retain the ability to perform end-to-end training remains an open question. In this paper, we present a novel framework for introducing declarative knowledge to neural network architectures in order to guide training and prediction. Our framework systematically compiles logical statements into computation graphs that augment a neural network without extra learnable parameters or manual redesign. We evaluate our modeling strategy on three tasks: machine comprehension, natural language inference, and text chunking. Our experiments show that knowledge-augmented networks can strongly improve over baselines, especially in low-data regimes.
- Publication:
-
arXiv e-prints
- Pub Date:
- June 2019
- DOI:
- 10.48550/arXiv.1906.06298
- arXiv:
- arXiv:1906.06298
- Bibcode:
- 2019arXiv190606298L
- Keywords:
-
- Computer Science - Machine Learning;
- Computer Science - Computation and Language;
- Statistics - Machine Learning
- E-Print:
- Accepted in ACL 2019. Minor fixes in Fig 4