Adjudication of coreference annotations via answer set optimisation
Abstract
We describe the first automatic approach for merging coreference annotations obtained from multiple annotators into a single gold standard. This merging is subject to certain linguistic hard constraints and optimisation criteria that prefer solutions with minimal divergence from annotators. The representation involves an equivalence relation over a large number of elements. We use Answer Set Programming to describe two representations of the problem and four objective functions suitable for different data-sets. We provide two structurally different real-world benchmark data-sets based on the METU-Sabanci Turkish Treebank and we report our experiences in using the Gringo, Clasp and Wasp tools for computing optimal adjudication results on these data-sets.
- Publication:
-
Journal of Experimental & Theoretical Artificial Intelligence
- Pub Date:
- July 2018
- DOI:
- 10.1080/0952813X.2018.1456793
- arXiv:
- arXiv:1802.00033
- Bibcode:
- 2018JETAI..30..525S
- Keywords:
-
- Coreference resolution;
- adjudication;
- answer set programming;
- Computer Science - Computation and Language;
- Computer Science - Artificial Intelligence
- E-Print:
- 3 tables, 10 figures, preliminary version presented at LPNMR 2017