Attention-over-Attention Neural Networks for Reading Comprehension
Abstract
Cloze-style queries are representative problems in reading comprehension. Over the past few months, we have seen much progress that utilizing neural network approach to solve Cloze-style questions. In this paper, we present a novel model called attention-over-attention reader for the Cloze-style reading comprehension task. Our model aims to place another attention mechanism over the document-level attention, and induces "attended attention" for final predictions. Unlike the previous works, our neural network model requires less pre-defined hyper-parameters and uses an elegant architecture for modeling. Experimental results show that the proposed attention-over-attention model significantly outperforms various state-of-the-art systems by a large margin in public datasets, such as CNN and Children's Book Test datasets.
- Publication:
-
arXiv e-prints
- Pub Date:
- July 2016
- DOI:
- 10.48550/arXiv.1607.04423
- arXiv:
- arXiv:1607.04423
- Bibcode:
- 2016arXiv160704423C
- Keywords:
-
- Computer Science - Computation and Language;
- Computer Science - Neural and Evolutionary Computing
- E-Print:
- 8+2 pages. accepted as a conference paper at ACL2017 (long paper)