Pre-Trained Text Embeddings for Enhanced Text-to-Speech Synthesis

Tomoki Hayashi, Shinji Watanabe, Tomoki Toda, Kazuya Takeda, Shubham Toshniwal, Karen Livescu

We propose an end-to-end text-to-speech (TTS) synthesis model that explicitly uses information from pre-trained embeddings of the text. Recent work in natural language processing has developed self-supervised representations of text that have proven very effective as pre-training for language understanding tasks. We propose using one such pre-trained representation (BERT) to encode input phrases, as an additional input to a Tacotron2-based sequence-to-sequence TTS model. We hypothesize that the text embeddings contain information about the semantics of the phrase and the importance of each word, which should help TTS systems produce more natural prosody and pronunciation. We conduct subjective listening tests of our proposed models using the 24-hour LJSpeech corpus, finding that they improve mean opinion scores modestly but significantly over a baseline TTS model without pre-trained text embedding input.

doi: 10.21437/Interspeech.2019-3177

Cite as: Hayashi, T., Watanabe, S., Toda, T., Takeda, K., Toshniwal, S., Livescu, K. (2019) Pre-Trained Text Embeddings for Enhanced Text-to-Speech Synthesis. Proc. Interspeech 2019, 4430-4434, doi: 10.21437/Interspeech.2019-3177

@inproceedings{hayashi19_interspeech,
  author={Tomoki Hayashi and Shinji Watanabe and Tomoki Toda and Kazuya Takeda and Shubham Toshniwal and Karen Livescu},
  title={{Pre-Trained Text Embeddings for Enhanced Text-to-Speech Synthesis}},
  year=2019,
  booktitle={Proc. Interspeech 2019},
  pages={4430--4434},
  doi={10.21437/Interspeech.2019-3177}
}