EmotiVoice - NetEase Youdao's open-source AI speech synthesis system

What is EmotiVoice?

EmotiVoice is an open-source text-to-speech (TTS) system from NetEase Youdao, supporting multiple languages, voices, and prompts. EmotiVoice supports English and Chinese, offers over 2000 voices, and can generate voices with different emotions (such as happiness, sadness, and anger) based on prompts. EmotiVoice features emotion synthesis and voice cloning capabilities, and provides a web interface and OpenAI-compatible APIs for easy user adoption and developer integration. It is suitable for various scenarios such as voice assistants and audiobooks.

Main functions of EmotiVoice

• Multilingual supportSupports both Chinese and English.
• A vast collection of timbresIt offers more than 2,000 different timbres.
• Emotional SynthesisIt supports the synthesis of speech containing a wide range of emotions, including happiness, excitement, sadness, and anger.
• Ease of useIt provides a simple and easy-to-use web interface and supports batch generation of script interfaces.
• Voice cloningSupports voice cloning.

EmotiVoice's technical principles

• Emotional and style controlStyle embedding involves embedding descriptions of emotion or style into the model, allowing it to generate speech with corresponding emotions or styles based on input prompts. During training, the model is trained with speech data containing various emotions and styles to better understand and generate speech with different emotions and styles. When generating speech, the model performs conditional generation based on the input text and emotion/style prompts, thus controlling the emotion and style.
• Multilingual and multi-voice supportThe model is trained using data from multiple languages to understand and generate speech in different languages. A unique embedding vector is trained for each speaker to generate speech for each individual speaker.
• Efficient reasoning and deploymentEmotiVoice can be quickly deployed using Docker containers, eliminating the need for manual installation and complex dependency configurations. EmotiVoice provides an OpenAI-compatible TTS API, facilitating integration and use within existing systems. Users can perform interactive speech synthesis via a web interface or generate speech in batches using a user script interface.
• Pre-trained models and fine-tuningThe model is pre-trained on large-scale speech data to learn common speech features and patterns. Users can fine-tune the pre-trained model according to their needs, such as adjusting the speech speed, pitch, or emotional intensity to generate speech that meets their requirements.

EmotiVoice project address

• GitHub repository:https://github.com/netease-youdao/EmotiVoice

Application scenarios of EmotiVoice

• Content creationGenerate audiobooks, podcasts, video dubbing, etc., supporting a variety of styles and emotions.
• Intelligent voice assistantIt is used in smart homes and in-vehicle systems to provide voice interaction and emotional feedback.
• EducationIt assists language learning by generating audio for online courses, enhancing the learning experience.
• Customer service systemUsed in intelligent customer service and voice response, providing multi-style voice services.
• Entertainment and GamesVoice acting for game characters, used for interactive entertainment, and to enhance immersion.