Chroma 1.0 - A real-time end-to-end speech dialogue model open-sourced by FlashLabs.

What is Chroma 1.0?

Chroma 1.0 is FlashLabs' first open-source real-time end-to-end speech dialogue model, combining low-latency interaction, high-fidelity personalized voice cloning, and strong conversational capabilities. The model achieves sub-second latency output by tightly coupling speech understanding and generation, employing a 1:2 text-to-audio token scheduling strategy. Requires only a few seconds of reference audio, it can highly reproduce the speaker's vocal characteristics, achieving a speaker similarity 10.96% higher than the human baseline. With only 4B parameters, the model performs exceptionally well in reasoning and spoken dialogue tasks, balancing efficiency and performance.

Main features of Chroma 1.0

• Real-time voice interactionIt supports low-latency voice dialogue, with end-to-end latency of less than 1 second, making it suitable for real-time interaction scenarios.
• High-fidelity voice cloningThe model only needs a few seconds of reference audio to achieve highly similar personalized speech synthesis, with a timbre reproduction accuracy 10.96% higher than the human baseline.
• Strong conversational skillsThe model possesses the ability to understand, reason, and engage in spoken dialogue, supporting complex dialogue tasks such as story logic and factual judgment.
• Streaming generationIt adopts a streaming output architecture, supports continuous dialogue, and generates content faster than real-time playback (RTF is 0.43).
• Multimodal fusionIt combines text and audio input, preserving paralinguistic information such as rhythm and intonation of speech, to achieve a more natural interaction.

Technical principles of Chroma 1.0

• Tightly Coupled Speech Understanding and GenerationIt tightly integrates the speech understanding module (Chroma Reasoner) with the speech generation modules (Chroma Backbone, Chroma Decoder, and Chroma Codec Decoder) to achieve low-latency streaming output through semantic state representation.
• 1:2 Text-to-Audio Token Scheduling StrategyDuring the generation process, each text token corresponds to two audio codebook tokens, enabling audio and text to be generated synchronously, significantly reducing latency.
• High-fidelity voice cloningBy embedding reference audio and corresponding text into the input sequence, the model can learn and reproduce the vocal characteristics of a specific speaker.
• Multimodal attention mechanismUsing multimodal rotation position coding (TM-RoPE) with cross-modal attention and temporal alignment, we ensure temporal alignment of speech and text, improving the naturalness of the conversation.
• Discrete acoustic representation and causal CNNThe model uses a discrete acoustic codebook to represent speech and performs waveform reconstruction through a causal convolutional neural network (Causal CNN), supporting real-time streaming output.

Chroma 1.0 project address

• GitHub repositoryhttps://github.com/FlashLabs-AI-Corp/FlashLabs-Chroma
• HuggingFace model library:https://huggingface.co/FlashLabs/Chroma-4B
• arXiv technical paper: https://arxiv.org/pdf/2601.11141

Application scenarios of Chroma 1.0

• Intelligent Customer ServiceIt provides users with real-time voice interaction services, quickly responds to customer needs, offers personalized voice answers, and enhances user experience.
• voice assistantIt can be integrated into smart homes and smart devices, enabling convenient operation through natural voice dialogue control.
• Virtual streamerUsed in news broadcasting, live streaming, and other fields, it generates high-fidelity audio, simulates the style of specific anchors, and enhances content diversity.
• Voice content creationIt assists content creators in generating high-quality audio content, such as audiobooks and audio stories, thereby improving creation efficiency.
• EducationIt provides language learners with personalized voice dialogue practice, real-time feedback on pronunciation and intonation, and improves language learning effectiveness.