Learning Context-Free Languages with Nondeterministic Stack RNNs
Abstract
We present a differentiable stack data structure that simultaneously and tractably encodes an exponential number of stack configurations, based on Lang's algorithm for simulating nondeterministic pushdown automata. We call the combination of this data structure with a recurrent neural network (RNN) controller a Nondeterministic Stack RNN. We compare our model against existing stack RNNs on various formal languages, demonstrating that our model converges more reliably to algorithmic behavior on deterministic tasks, and achieves lower cross-entropy on inherently nondeterministic tasks.
- Publication:
-
arXiv e-prints
- Pub Date:
- October 2020
- DOI:
- 10.48550/arXiv.2010.04674
- arXiv:
- arXiv:2010.04674
- Bibcode:
- 2020arXiv201004674D
- Keywords:
-
- Computer Science - Computation and Language
- E-Print:
- 13 pages, 5 figures. Published at CoNLL 2020. This revision fixes a typo