Qwen3-TTS - A series of open-source speech generation models from Alibaba Tongyi

What is Qwen3-TTS?

Qwen3-TTS is a series of open-source speech generation models from Qwen, boasting powerful capabilities in timbre cloning, creation, and voice control. Based on the innovative Qwen3-TTS-Tokenizer-12Hz multi-codebook speech encoder, the model achieves efficient speech compression and high-fidelity reproduction. Employing Dual-Track modeling, it supports low-latency streaming generation, requiring only a one-character wait for the first audio packet. The model covers 10 mainstream languages (Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, and Italian) and various dialects, possessing intelligent text understanding capabilities and adaptively adjusting tone, rhythm, and emotion. The entire Qwen3-TTS multi-codebook model series is open-source, available in 1.7B and 0.6B sizes to meet different performance and efficiency needs, providing developers and users with comprehensive speech generation functionality.

Main functions of Qwen3-TTS

• Timbre CloningThe model can clone the timbre of a specific speaker using a small amount of reference audio, achieving highly similar speech synthesis.
• Sound creationIt supports generating customized timbre images through natural language descriptions, allowing users to freely define acoustic attributes, personas, and background information to create unique timbres.
• Voice controlIt supports users to flexibly adjust multi-dimensional acoustic attributes such as timbre, emotion, and rhythm through commands to achieve accurate speech expression.
• Multilingual supportThe model covers 10 mainstream languages (such as Chinese, English, Japanese, etc.) and multiple dialects, meeting the needs of global applications.
• Low-latency streaming generationBased on innovative Dual-Track modeling, it achieves ultra-fast bidirectional streaming generation, with the first packet audio waiting time being only one character and the end-to-end synthesis latency as low as 97ms.
• contextual understandingThe model possesses powerful text semantic understanding capabilities, and can automatically adjust tone, rhythm, and emotion based on the input text to adapt to different scenarios.
• High-fidelity reproductionRelying on the self-developed Qwen3-TTS-Tokenizer-12Hz, it fully preserves the secondary language information and acoustic environment characteristics, achieving efficient and high-fidelity speech reproduction.

The technical principles of Qwen3-TTS

• Qwen3-TTS-Tokenizer-12HzBased on a multi-codebook speech encoder, it can efficiently compress speech signals and perform high-dimensional semantic modeling. The encoder fully preserves paralinguistic information (such as intonation, rhythm, and emotion) and acoustic environment features, and achieves high-speed, high-fidelity speech reconstruction through a lightweight non-DiT (Discrete Inverse Transform) architecture.
• Dual-Track ModelingCombining streaming and non-streaming generation methods, a single model supports both modes simultaneously. It achieves extremely low-latency streaming generation, outputting the first audio packet almost immediately after inputting a single word, with end-to-end synthesis latency as low as 97ms, meeting the needs of real-time interactive scenarios.
• Discrete Multicodebook LM ArchitectureThe model adopts a discrete multi-codebook language model (LM) architecture to achieve end-to-end modeling of all speech information. It avoids the information bottleneck and cascade error in the traditional LM+DiT scheme, and significantly improves the model's versatility, generation efficiency, and performance ceiling.
• Natural language instruction driverThe model supports speech generation driven by natural language commands, allowing users to control attributes such as timbre, emotion, and rhythm through simple text descriptions. It deeply integrates text semantic understanding, adaptively adjusting tone, rhythm, and emotion to achieve a human-like expression where "what you think is what you hear."

Qwen3-TTS project address

• GitHub repositoryhttps://github.com/QwenLM/Qwen3-TTS
• HuggingFace model libraryhttps://huggingface.co/collections/Qwen/qwen3-tts

Application scenarios of Qwen3-TTS

• Intelligent voice assistantIt provides natural voice interaction for smart home devices and in-vehicle systems, supporting multiple languages and dialects to enhance the user experience.
• Content creationIt quickly converts text into natural speech, supports a variety of timbres and emotional expressions, and is suitable for audiobooks and video dubbing.
• EducationIt provides multilingual and multi-voice audio output for language learning and online teaching, enhancing learning effectiveness.
• Games and entertainmentGenerate personalized voices for game characters, supporting emotional and intonation adjustments to enhance game immersion.
• Customer service and supportProvides multilingual and multi-voice support for intelligent customer service and voice broadcasting in public places, improving service efficiency.