Natural Language Processing with Small Feed-Forward Networks
Abstract
We show that small and shallow feed-forward neural networks can achieve near state-of-the-art results on a range of unstructured and structured language processing tasks while being considerably cheaper in memory and computational requirements than deep recurrent models. Motivated by resource-constrained environments like mobile phones, we showcase simple techniques for obtaining such small neural network models, and investigate different tradeoffs when deciding how to allocate a small memory budget.
- Publication:
-
arXiv e-prints
- Pub Date:
- August 2017
- DOI:
- 10.48550/arXiv.1708.00214
- arXiv:
- arXiv:1708.00214
- Bibcode:
- 2017arXiv170800214B
- Keywords:
-
- Computer Science - Computation and Language;
- Computer Science - Neural and Evolutionary Computing;
- 68T50;
- I.2.7
- E-Print:
- EMNLP 2017 short paper