Emotions are Subtle: Learning Sentiment Based Text Representations Using Contrastive Learning
Abstract
Contrastive learning techniques have been widely used in the field of computer vision as a means of augmenting datasets. In this paper, we extend the use of these contrastive learning embeddings to sentiment analysis tasks and demonstrate that fine-tuning on these embeddings provides an improvement over fine-tuning on BERT-based embeddings to achieve higher benchmarks on the task of sentiment analysis when evaluated on the DynaSent dataset. We also explore how our fine-tuned models perform on cross-domain benchmark datasets. Additionally, we explore upsampling techniques to achieve a more balanced class distribution to make further improvements on our benchmark tasks.
- Publication:
-
arXiv e-prints
- Pub Date:
- December 2021
- DOI:
- 10.48550/arXiv.2112.01054
- arXiv:
- arXiv:2112.01054
- Bibcode:
- 2021arXiv211201054M
- Keywords:
-
- Computer Science - Computation and Language
- E-Print:
- 9 pages, 5 figures