Augmenting Compositional Models for Knowledge Base Completion Using Gradient Representations
Abstract
Neural models of Knowledge Base data have typically employed compositional representations of graph objects: entity and relation embeddings are systematically combined to evaluate the truth of a candidate Knowedge Base entry. Using a model inspired by Harmonic Grammar, we propose to tokenize triplet embeddings by subjecting them to a process of optimization with respect to learned well-formedness conditions on Knowledge Base triplets. The resulting model, known as Gradient Graphs, leads to sizable improvements when implemented as a companion to compositional models. Also, we show that the "supracompositional" triplet token embeddings it produces have interpretable properties that prove helpful in performing inference on the resulting triplet representations.
- Publication:
-
arXiv e-prints
- Pub Date:
- November 2018
- DOI:
- 10.48550/arXiv.1811.01062
- arXiv:
- arXiv:1811.01062
- Bibcode:
- 2018arXiv181101062L
- Keywords:
-
- Computer Science - Computation and Language
- E-Print:
- 10 pages, 2 figures, To appear in proceedings of the Society for Computation in Linguistics (SCIL 2019)