Self-supervised Context-aware Style Representation for Expressive Speech Synthesis
Abstract
Expressive speech synthesis, like audiobook synthesis, is still challenging for style representation learning and prediction. Deriving from reference audio or predicting style tags from text requires a huge amount of labeled data, which is costly to acquire and difficult to define and annotate accurately. In this paper, we propose a novel framework for learning style representation from abundant plain text in a self-supervised manner. It leverages an emotion lexicon and uses contrastive learning and deep clustering. We further integrate the style representation as a conditioned embedding in a multi-style Transformer TTS. Comparing with multi-style TTS by predicting style tags trained on the same dataset but with human annotations, our method achieves improved results according to subjective evaluations on both in-domain and out-of-domain test sets in audiobook speech. Moreover, with implicit context-aware style representation, the emotion transition of synthesized audio in a long paragraph appears more natural. The audio samples are available on the demo web.
- Publication:
-
arXiv e-prints
- Pub Date:
- June 2022
- DOI:
- 10.48550/arXiv.2206.12559
- arXiv:
- arXiv:2206.12559
- Bibcode:
- 2022arXiv220612559W
- Keywords:
-
- Computer Science - Sound;
- Computer Science - Artificial Intelligence;
- Computer Science - Computation and Language;
- Electrical Engineering and Systems Science - Audio and Speech Processing
- E-Print:
- Accepted by Interspeech 2022