IndexTTS - A text-to-speech model launched by Bilibili, supporting pinyin correction for Chinese character pronunciation.

What is IndexTTS?

IndexTTS is an industrial-grade, controllable text-to-speech (TTS) system launched by Bilibili. Developed based on the XTTS and Tortoise models and incorporating GPT-style generation techniques, it efficiently converts text into natural and fluent speech. IndexTTS's strengths lie in its ability to process Chinese text, supporting pinyin correction for Chinese character pronunciation and allowing precise control of pauses through punctuation, effectively addressing pronunciation issues with polyphonic characters and long-tailed characters. It employs a hybrid modeling approach, combining Chinese characters and pinyin to optimize speech generation. In terms of performance, IndexTTS boasts a word error rate (WER) of only 1.3%, a speaker similarity (SS) of 0.776, and a subjective sound quality score (MOS) of 4.01, demonstrating excellent performance. It is trained using a large amount of data, including 25,000 hours of Chinese audio and 9,000 hours of English audio, ensuring high-quality sound and timbre.

The latest self-developed IndexTTS2 next-generation speech synthesis model can achieve precise duration control and emotion reproduction, supports independent adjustment of timbre and emotion, and can generate natural and fluent speech in multiple languages and styles. It is widely used in video dubbing, language learning and other fields. The model has achieved industry-leading levels in both emotional expression and text accuracy.

Main functions of IndexTTS

• Pinyin correction and pause controlWhen processing Chinese text, IndexTTS can correct the pronunciation of Chinese characters through pinyin and use punctuation marks to precisely control pauses at any position, making the speech output more natural and fluent.
• Sound quality optimizationThe introduction of a Conformer-based conditional encoder and BigVGAN2 speech decoder significantly improved sound quality and timbre similarity, achieving a MOS score of 4.01.
• Multilingual supportCurrently supports Chinese and English, with plans to expand to more languages in the future.

The technical principles of IndexTTS

• Hybrid modeling methodsIn Chinese scenarios, IndexTTS employs a hybrid modeling approach combining characters and pinyin. By combining Chinese characters and pinyin, the system can quickly correct pronunciation errors of polyphonic characters and long-tailed characters, improving the accuracy and controllability of speech synthesis.
• Speech encoding and decoding optimizationIndexTTS introduces a Conformer-based conditional encoder and a BigVGAN2 speech decoder. The Conformer encoder better captures the timbre features of speech, while the BigVGAN2 decoder improves sound quality and timbre similarity.
• Zero-sample speech cloningIndexTTS excels in zero-sample speech cloning, generating high-quality speech even without target speech samples. It achieves a speaker similarity (SS) of up to 0.776 and a MOS score of 4.01, far exceeding existing systems.
• Efficient training and reasoningThe system has been trained on tens of thousands of hours of data, making the training process simple and stable, and its inference speed fast. Compared with systems such as XTTS, CosyVoice2, Fish-Speech, and F5-TTS, IndexTTS performs better in terms of naturalness, content consistency, and zero-shot speech cloning.

IndexTTS project address

• IndexTTS:
  • Github repositoryhttps://github.com/index-tts/index-tts
  • arXiv technical paper: https://arxiv.org/pdf/2502.05512
• IndexTTS2:
  • Project official websitehttps://index-tts.github.io/index-tts2.github.io/
  • HuggingFace model library:https://huggingface.co/IndexTeam/IndexTTS-2
  • arXiv technical paper: https://arxiv.org/pdf/2506.21619

Application scenarios of IndexTTS

• Content creation and video dubbingIndexTTS can quickly generate natural and fluent audio content, helping video creators save recording time and quickly complete video dubbing.
• Audiobooks and Online EducationIt supports high-quality speech synthesis in both Chinese and English, providing natural voice reading services for audiobooks and online education platforms, thus enhancing the user experience.
• Intelligent customer service and voice assistantIndexTTS's zero-sample speech cloning capability and high naturalness performance make it suitable for intelligent customer service and voice assistant scenarios, and it can quickly adapt to different speech styles and speeds.
• Multimedia and EntertainmentIn the multimedia and entertainment field, IndexTTS can be used to generate personalized voice content, such as game voice-overs and virtual character dialogues, providing users with a more immersive experience.
• Accessibility technologyWith its efficient speech synthesis capabilities and low error rate, IndexTTS can provide voice assistance to visually impaired individuals, helping them to better access information.