AB
AB
AiBoss

All Categories

project

Maya1 - An open-source speech synthesis model from the Maya Research team.

Maya1 is an open-source AI speech synthesis model from the Maya Research team, designed specifically to generate emotionally expressive speech. The model uses natural language descriptions to design voices and supports over 20 emotional expressions, such as laughter, crying, etc.

What is Maya1?

Maya1 is an open-source AI speech synthesis model from the Maya Research team, designed specifically to generate emotionally rich speech. The model designs its voice using natural language descriptions, supports over 20 emotional expressions such as laughter, crying, and sighing, and can stream audio in real time. Based on a 3 billion-parameter Transformer architecture and the SNAC neural codec, the model outputs 24kHz audio with high quality and low latency. It is suitable for various scenarios such as game voice-over, podcast production, and voice assistant development, making AI voices more human-like and expressive.

Main functions of Maya1

  • Natural Language Voice DesignUsers can define voice characteristics through simple natural language descriptions (such as "30-year-old American woman with a gentle and sincere voice") without the need for complex parameter adjustments.
  • Rich emotional expressionSupports over 20 emotions, such as laugh, cry, and sigh, and can be expressed through emotion tags in the text (e.g., <laugh>Precisely control emotional expression.
  • Real-time streamingIt uses the SNAC neural codec to support real-time audio generation with low latency (approximately 100 milliseconds), making it suitable for scenarios requiring instant feedback, such as voice assistants and game dialogues.
  • Efficient deploymentBased on a lightweight Transformer architecture with 3 billion parameters, it can run on a single GPU, supports the vLLM inference framework, and is suitable for high-concurrency scenarios.

Maya1's technical principles

  • ArchitectureMaya1 is based on a Transformer architecture (similar to Llama) with 3 billion parameters, used to generate audio token sequences for the SNAC codec, rather than directly generating waveforms.
  • SNAC codecBy using multi-scale layered compression (approximately 12Hz/23Hz/47Hz), audio is efficiently encoded into 7-token frames, achieving high-quality audio output at a low bit rate (approximately 0.98kbps).
  • Training processPre-trainingUsing large-scale English speech data, covering a variety of accents and speaking speeds.Based on studio-quality voice samples, more than 20 emotion and identity tags were labeled.
  • Sound description: Using XML attribute-based natural language description (e.g. <description="...">This prevents the model from "reading" the description aloud.
  • Inference optimizationIt supports vLLM engine integration and, combined with the Automatic Prefix Caching (APC) mechanism, significantly reduces the computational cost of repeated generation. It is also compatible with WebAudio circular buffers, facilitating real-time playback on the browser side.

Maya1's project address

  • HuggingFace model libraryhttps://huggingface.co/maya-research/maya1

Application scenarios of Maya1

  • Game developmentGenerate emotional dialogue for game characters to enhance immersion, such as having NPCs express sneer or anger in their conversations.
  • Podcasts and AudiobooksAutomatic voice-over, supporting multi-character dialogue and emotional expression, eliminating the need for professional voice actors and enhancing content appeal.
  • AI voice assistant: To create a natural and emotional voice interaction experience, allowing the assistant to express emotions such as sympathy and joy when responding.
  • Short video creationQuickly generate emotionally charged narration to enhance the expressiveness of videos and the audience's immersion.
  • Accessibility applicationsTo make screen readers more user-friendly, using warm and natural voices to help visually impaired people better understand content.
Back