AB
AB
AiBoss

All Categories

project

ChatTTS - An open-source generative speech synthesis model for dialogue.

ChatTTS is a text-to-speech (TTS) model designed specifically for conversational scenarios, supporting both Chinese and English. It is trained on approximately 100,000 hours of Chinese and English data and can generate high-quality, natural, and fluent conversational speech.

What is ChatTTS?

ChatTTS is a text-to-speech (TTS) model designed specifically for conversational scenarios, supporting both Chinese and English. Trained on approximately 100,000 hours of Chinese and English data, it generates high-quality, natural, and fluent conversational speech. Optimized for conversational tasks, ChatTTS achieves more natural and fluent speech synthesis, supports multiple speakers, and possesses fine-grained control capabilities, predicting and controlling prosodic features such as laughter and pauses, surpassing most open-source TTS models.

ChatTTS Features

  • Text-to-speechChatTTS can convert user-input text information into natural and fluent speech output in real time, and is suitable for multiple language environments.
  • Multilingual supportIn addition to supporting Chinese, ChatTTS can also handle English text, enabling it to serve a wider range of users.
  • Emotional and rhythmic adjustmentChatTTS can not only convert text, but also adjust the emotional tone and rhythm of the voice according to the text content, such as speech rate, intonation, and pauses, making the voice closer to the natural rhythm of real human speech.
  • Voice character selectionUsers can choose the most suitable voice from multiple preset voice characters according to the needs of the application scenario, increasing the personalization and expressiveness of the voice.
  • Interactive Web InterfaceThrough an intuitive web interface, users can directly enter text in their browser and receive voice output without writing any code.
  • Real-time voice interactionChatTTS supports real-time speech synthesis, making it ideal for dialogue systems and interactive applications that require immediate feedback.
  • Exporting audio filesUsers can export the synthesized speech into common audio file formats for easy editing, sharing, or use as part of multimedia content.
  • Integration and compatibilityChatTTS supports integration into various platforms and applications, and can be seamlessly integrated into multiple environments such as web applications, mobile applications, and desktop software.
  • Emotion tagging systemChatTTS supports embedding emotion markers in text, allowing users to finely control the emotional expression of speech output, such as inserting [laugh] markers in text to simulate laughter.

ChatTTS official website entrance

How to run ChatTTS

Method 1: Experience the Demo Online

Regular users can directly experience the online ChatTTS WebUI demo provided by the community on ModelScope and Hugging Face.

Method 2: Local Deployment and Operation

  1. Installation EnvironmentMake sure you have Python and Git installed on your computer.
  2. SDK DownloadInstall ModelScope and download the SDK model. 
    #安装ModelScope
pip install modelscope
    #SDK模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('pzc163/chatTTS')
  3. Get the source codeClone ChatTTS source code from ModelScope's code repository to your local machine using the Git version control system. 
    #Git模型下载
git clone https://www.modelscope.cn/pzc163/chatTTS.git
  4. Install dependenciesNavigate to the project directory and use the pip command to install the required Python dependencies. 
    pip install -r requirement.txt
pip install Ipython
pip install soundfile
  5. Model InferenceYou can use the free computing power from the Moda community to complete model inference. 
    from ChatTTS import Chat
from IPython.display import Audio
#下载模型
from modelscope import snapshot_download

model_dir = snapshot_download('pzc163/chatTTS')

chat = Chat()
chat.load_models(source='local', local_path=model_dir)

texts = ["你好,我是ChatTTS,很高兴认识大家",]

wavs = chat.infer(texts, use_decoder=True)
Audio(wavs[0], rate=24_000, autoplay=True)

# save audio
import soundfile as sf
audio_data = wavs[0]
if len(audio_data.shape) > 1: 
 audio_data = audio_data.flatten()

output_file = './output_audio2.wav'
sf.write(output_file, audio_data, 24000)
print(f"Audio saved to {output_file}")
  6. Build and run the WebUI 
    git clone https://www.modelscope.cn/studios/AI-ModelScope/ChatTTS-demo.git
cd ChatTTS
pip install -r requirements.txt
python app.py

Application scenarios of ChatTTS

  • Virtual assistants and customer service robotsChatTTS can provide natural and fluent voice output for virtual assistants and online customer service robots, enhancing the user experience.
  • Audiobooks and eBooksIt converts text content into speech, providing voice reading functionality for audiobooks and ebooks, making it convenient for users to listen while commuting or doing housework.
  • Social media and content creationChatTTS can generate engaging audio content on social media platforms or in content creation, increasing interactivity and fun.
  • News and podcastsAutomatically converts press releases or blog posts into audio for podcasts or news broadcasts.
  • AccessibilityIt provides voice assistance to visually impaired people or users with reading difficulties, enabling them to obtain information by listening.
Back