AB
AB
AiBoss

All Categories

project

Speech-02 - MiniMax's next-generation text-to-speech model

Speech-02 is a next-generation text-to-speech (TTS) model from MiniMax. Based on a regressive Transformer architecture, the model achieves zero-shot speech cloning, generating highly similar target speech from just a few seconds of reference speech...

What is Speech-02?

Speech-02 is MiniMax's next-generation text-to-speech (TTS) model. Based on a regressive Transformer architecture, it achieves zero-shot speech cloning, generating highly similar target speech with just a few seconds of reference speech. The Flow-VAE architecture enhances the information representation capabilities of speech generation, improving the quality and similarity of synthesized speech. Speech-02 is available in two versions: Speech-02-HD, designed for high-fidelity applications such as voice-over and audiobooks, eliminating rhythm inconsistencies and maintaining clear sound quality; and Speech-02-Turbo, optimized for real-time performance, balancing ultra-low latency with superior sound quality, suitable for interactive applications. The Speech-02 model is available on the MiniMax Audio platform and the MiniMax API platform.

Main functions of Speech-02

  • Zero-sample speech cloningIt generates highly similar target speech with only a few seconds of reference speech.
  • High-quality speech synthesisGenerates natural and fluent speech, supporting multiple languages and dialects.
  • Multilingual supportSupports 32 languages, is proficient in Chinese, English, Cantonese and other languages, and can switch between languages.
  • Personalized voice generationUsers provide sample audio, and the model learns from it to generate personalized speech.
  • Emotional controlIt supports multiple emotions (such as happiness, sadness, etc.) and generates speech based on text descriptions.

Technical Principles of Speech-02

  • Autoregressive Transformer ArchitectureBased on an autoregressive Transformer architecture, it generates speech with better prosody, intonation, and overall naturalness. The autoregressive model generates speech features one by one during the generation process, ensuring that the generated speech is more natural and coherent.
  • Zero-sample speech cloningA learnable speaker encoder is introduced, which specifically learns the most useful vocal features for synthesized speech, such as the speaker's unique pronunciation habits. The model generates highly similar target speech from only a few seconds of reference speech.
  • Flow-VAE ArchitectureBased on invertible mapping transformation of the latent space, it more accurately captures complex patterns in data. The Flow-VAE architecture enhances the information representation capability in the speech generation process, improving the overall quality and similarity of synthesized speech.
  • T2V frameThe T2V framework combines open-ended natural language descriptions with structured tagging information to achieve highly flexible and controllable timbre generation. Users guide the model to generate speech with specific timbres and emotions based on textual descriptions.

Speech-02 project address

Application scenarios of Speech-02

  • Intelligent voice assistantTo provide a natural and smooth human-computer interaction experience for smart devices and improve user satisfaction.
  • Audiobooks and DubbingProduction of high-quality audiobooks, voice-over for commercials, etc.
  • Social media and entertainmentIt provides personalized voice generation in scenarios such as social media, live streaming, and singing and chatting, enhancing user interactivity and entertainment.
  • Education and children's toysIt can be applied to educational learning machines, children's toys, etc., to provide a more vivid and interesting learning experience.
  • Smart hardware integrationIt can be integrated with hardware devices such as smart speakers and smart car cockpits to enhance the intelligence level of the devices.
Back