XPhoneBERT : A Pre-trained Multilingual Model for Phoneme Representations for Text-to-Speech

@inproceedings{xphonebert,
title     = {{XPhoneBERT: A Pre-trained Multilingual Model for Phoneme Representations for Text-to-Speech}},
author    = {Linh The Nguyen and Thinh Pham and Dat Quoc Nguyen},
booktitle = {Proceedings of the 24th Annual Conference of the International Speech Communication Association (INTERSPEECH)},
year      = {2023},
pages     = {5506--5510}
}

Using XPhoneBERT with transformers

Installation

• Install transformers with pip: pip install transformers, or install transformers from source.
• Install text2phonemesequence: pip install text2phonemesequence
  Our text2phonemesequence package is to convert text sequences into phoneme-level sequences, employed to construct our multilingual phoneme-level pre-training data. We build text2phonemesequence by incorporating the CharsiuG2P and the segments toolkits that perform text-to-phoneme conversion and phoneme segmentation, respectively.
• Notes
  • Initializing text2phonemesequence for each language requires its corresponding ISO 639-3 code. The ISO 639-3 codes of supported languages are available at HERE.
  • text2phonemesequence takes a word-segmented sequence as input. And users might also perform text normalization on the word-segmented sequence before feeding into text2phonemesequence. When creating our pre-training data, we perform word and sentence segmentation on all text documents in each language by using the spaCy toolkit, except for Vietnamese where we employ the VnCoreNLP toolkit. We also use the text normalization component from the NVIDIA NeMo toolkit for English, German, Spanish and Chinese, and the Vinorm text normalization package for Vietnamese.

Pre-trained model

Example usage