AB
AB
AiBoss

All Categories

project

FireRedChat - A full-duplex voice interaction system launched by Xiaohongshu (Little Red Book)

FireRedChat is a full-duplex voice interaction system developed by the Xiaohongshu Intelligent Audio Team. It features real-time two-way dialogue capabilities and supports controlled interruption. It adopts a modular design, including a transcription control module, an interaction module, and a dialogue module...

What is FireRedChat?

FireRedChat is a full-duplex voice interaction system developed by the Xiaohongshu Intelligent Audio Team. It features real-time two-way dialogue capabilities and supports controlled interruption. Employing a modular design, it includes a transcription control module, an interaction module, and a dialogue manager, supporting cascaded and semi-cascaded architectures for flexible deployment. The system leverages LiveKit RTC Server for real-time communication, coupled with an AI-Agent Bot Server to handle intelligent agent responses, and provides a user interface via WebUI. It also includes a Redis Server for multi-node hosting, and TTS and ASR Servers for speech synthesis and automatic speech recognition, respectively.

FireRedChat's main functions

  • Full-duplex voice interactionIt supports real-time two-way dialogue between users and AI agents, allowing both parties to speak simultaneously and enabling controlled interruptions, thus improving the smoothness of interaction.
  • Privacy protection and privatization deploymentThe system supports full self-management, does not rely on external APIs, ensures data security, and allows users to control the deployment environment independently.
  • Modular designIt consists of multiple modules, including transcription control, interaction modules, and a dialogue manager, and supports flexible cascaded and semi-cascaded architectures, making it easy to customize and extend.
  • Low-latency communicationIt achieves real-time communication based on LiveKit RTC Server, coupled with a high-efficiency processing module to ensure low-latency interaction, approaching industrial-grade standards.
  • Speech activity detection and semantic analysisIt employs streaming personalized speech activity detection (pVAD) and semantic end detection (EoT) to effectively suppress background noise, accurately mark the main speaker's speech segments, and improve the success rate of user interruption and the naturalness of the dialogue.

FireRedChat's technical principles

  • Real-time communication technologyIt uses LiveKit RTC Server as its core to achieve low-latency real-time audio/video communication and supports simultaneous interaction by multiple users.
  • Smart Agent ResponseIt processes user input through the AI-Agent Bot Server and combines it with natural language processing technology to generate intelligent and natural voice responses.
  • Speech recognition and synthesisThe system integrates an ASR Server to enable automatic speech recognition, converting user speech into text; the TTS Server then converts the proxy's text responses into speech output.
  • Voice activity detectionIt utilizes streaming personalized speech activity detection (pVAD) technology to accurately identify speech segments of the main speaker and suppress background noise and non-target speakers.
  • Semantic End DetectionSemantic analysis helps determine whether a user has finished speaking, avoiding misjudgments caused by pauses in speech and improving the naturalness of the interaction.
  • Modular architectureThe system consists of multiple independent modules that work together, supporting flexible cascading and semi-cascading deployments, and facilitating expansion and maintenance.
  • Data persistence and hosting: Utilize Redis Server to achieve cross-instance data persistence, support multi-node hosting, and ensure high availability and stability of the system.

FireRedChat project address

  • Gtihub warehouse: https://github.com/FireRedTeam/FireRedChat
  • arXiv technical paper: https://arxiv.org/pdf/2509.06502
  • Online experiencehttps://fireredteam.github.io/demos/firered_chat

Application scenarios of FireRedChat

  • Intelligent Customer ServiceProvide users with real-time voice support, quickly respond to customer issues, and improve service efficiency and customer satisfaction.
  • Virtual AssistantIn smart home and smart office scenarios, it serves as the core of voice interaction, enabling functions such as device control and information retrieval.
  • EducationUsed on online education platforms to provide real-time interactive voice teaching and enhance the learning experience.
  • Financial industryIt provides secure and efficient voice interaction services in scenarios such as financial consulting and transaction assistance.
  • HealthcareIt assists in medical consultations and remote diagnosis, improving the convenience of medical services through voice interaction.
  • Government AffairsIn scenarios such as government hotlines and public services, intelligent voice services are provided to improve government efficiency.
Back