Comparison of Current Approaches to Lemmatization: A Case Study in Estonian
Abstract
This study evaluates three different lemmatization approaches to Estonian -- Generative character-level models, Pattern-based word-level classification models, and rule-based morphological analysis. According to our experiments, a significantly smaller Generative model consistently outperforms the Pattern-based classification model based on EstBERT. Additionally, we observe a relatively small overlap in errors made by all three models, indicating that an ensemble of different approaches could lead to improvements.
- Publication:
-
arXiv e-prints
- Pub Date:
- April 2024
- DOI:
- 10.48550/arXiv.2404.15003
- arXiv:
- arXiv:2404.15003
- Bibcode:
- 2024arXiv240415003D
- Keywords:
-
- Computer Science - Computation and Language
- E-Print:
- 6 pages, 2 figures