Hypothesis Only Baselines in Natural Language Inference
Abstract
We propose a hypothesis only baseline for diagnosing Natural Language Inference (NLI). Especially when an NLI dataset assumes inference is occurring based purely on the relationship between a context and a hypothesis, it follows that assessing entailment relations while ignoring the provided context is a degenerate solution. Yet, through experiments on ten distinct NLI datasets, we find that this approach, which we refer to as a hypothesis-only model, is able to significantly outperform a majority class baseline across a number of NLI datasets. Our analysis suggests that statistical irregularities may allow a model to perform NLI in some datasets beyond what should be achievable without access to the context.
- Publication:
-
arXiv e-prints
- Pub Date:
- May 2018
- DOI:
- 10.48550/arXiv.1805.01042
- arXiv:
- arXiv:1805.01042
- Bibcode:
- 2018arXiv180501042P
- Keywords:
-
- Computer Science - Computation and Language
- E-Print:
- Accepted at *SEM 2018 as long paper. 12 pages