Learning Meta-Embeddings by Using Ensembles of Embedding Sets
Abstract
Word embeddings -- distributed representations of words -- in deep learning are beneficial for many tasks in natural language processing (NLP). However, different embedding sets vary greatly in quality and characteristics of the captured semantics. Instead of relying on a more advanced algorithm for embedding learning, this paper proposes an ensemble approach of combining different public embedding sets with the aim of learning meta-embeddings. Experiments on word similarity and analogy tasks and on part-of-speech tagging show better performance of meta-embeddings compared to individual embedding sets. One advantage of meta-embeddings is the increased vocabulary coverage. We will release our meta-embeddings publicly.
- Publication:
-
arXiv e-prints
- Pub Date:
- August 2015
- DOI:
- 10.48550/arXiv.1508.04257
- arXiv:
- arXiv:1508.04257
- Bibcode:
- 2015arXiv150804257Y
- Keywords:
-
- Computer Science - Computation and Language
- E-Print:
- 10 pages, 6 figures