Qwen-Audio-Agent - Alibaba's open-source real-time voice agent framework

What is Qwen-Audio-Agent?

Qwen-Audio-Agent is an open-source real-time speech agent framework launched by Alibaba's speech AI team, built on the Qwen-Audio-3.0-Realtime model. As a unified real-time speech entry layer, the framework allows users to engage in full-duplex conversations with backend agents such as OpenCode, OpenClaw, and Codex through natural speech. It supports simultaneous listening and speaking, interruption at any time, and delegation of complex tasks to the backend for execution, returning the results later.

Main functions of Qwen-Audio-Agent

• Full-duplex real-time voiceIt supports continuous speaking and natural interruptions. Users can supplement or correct their needs at any time while the Agent is performing tasks, without waiting for the previous round to end.
• Agent Ecosystem AccessIt is compatible with mainstream agents such as OpenCode, OpenClaw, Qoder, Hermes, CodeBuddy, and Codex, and supports the ACP stdio protocol to extend to more backends.
• Task delegation and contextual coherenceSimple questions are answered instantly by the real-time voice front end, while complex tasks automatically pass the context to the backend Agent Runtime for execution, and the results are broadcast back to the conversation in real time.
• Multi-form interactive interfaceIt offers a TUI terminal interface, a WebUI web version, and a macOS desktop voice floating ball (supporting two appearances: a flowing light sound wave ball and a liquid gradient ball).
• Configuration Management:pass qwenaudio config Quickly configure the DashScope API Key and backend Agent protocol, and switch between different Agent backends with one click.

The technical principle of Qwen-Audio-Agent

• Full-duplex real-time voice interaction architectureThe core speech layer of Qwen-Audio-Agent is based on the Qwen-Audio-3.0-Realtime model, employing a full-duplex communication mechanism to enable simultaneous listening and speaking. The architecture allows users to interrupt the Agent's broadcast or execution at any time; the system will truncate the current output stream in real time and re-parse the new speech input, simulating the natural alternation rhythm of a real conversation.
• Task delegation model with front-end and back-end separationThe system adopts a decoupled two-tier architecture: a real-time voice front-end and an Agent Runtime back-end. The front-end is responsible for real-time transcription, semantic understanding, and immediate response of the voice signal. When a deep task such as searching, reasoning, or code modification is detected, the front-end delegates the current dialogue context to the back-end Agent for execution through a standardized interface. After completing the calculation, the back-end sends the result back to the front-end in text or voice format for unified announcement, avoiding complex tasks from blocking the real-time interaction flow.
• Context connection and state managementIn multi-turn continuous dialogues, the system maintains a unified dialogue state machine. User interruptions, additions, or direction changes do not clear the existing context but are incrementally appended to the current session state. When a task is delegated to a background agent, the relevant context is serialized and passed; the results returned by the agent are automatically integrated into the current session, ensuring that the user remains in a coherent semantic environment while chatting and working simultaneously.

Follow us on WeChat and reply with "open source",join inAI open source project discussion group

How to use Qwen-Audio-Agent

• Environmental preparationEnsure that you have a Node.js environment installed and a DashScope API Key ready.
• Global installation:implement npm install -g qwen-audio-agent Complete the installation of command-line tools.
• Create configuration:run qwenaudio configEnter the DashScope API Key and select the backend Agent protocol (e.g., ...). opencode）。
• Launch TUI:implement qwenaudio tui Open the terminal's voice interaction interface and start a real-time conversation.
• Launch WebUI:implement qwenaudio webui Open the web version of the voice interaction interface in your browser.
• Desktop version experienceExecute after cloning the source code npm install && npm run desktop Launch the macOS desktop floating ball, which will always reside in the corner of the screen and can be activated by voice at any time.

Qwen-Audio-Agent's core advantages

• Natural interactive experienceFull-duplex voice makes collaboration closer to real-life conversations, reducing input costs and improving communication efficiency and emotional value.
• Zero migration costIt does not replace the existing Agent, but directly reuses the user's existing toolchain, project context and permission system.
• Clear architecture decouplingThe real-time voice front-end and the Agent back-end are separated, each performing its own function, which facilitates independent iteration and expansion.
• Multi-platform coverageThe system offers three forms: TUI, WebUI, and desktop floating ball, to adapt to different work scenarios and usage habits.
• Open source and scalableBased on the ACP standard protocol, developers can freely integrate custom Agent backends and business logic.

Qwen-Audio-Agent project address

• GitHub repository：https://github.com/QwenAudio/qwen-audio-agent

Comparison of Qwen-Audio-Agent with similar competing products

Application scenarios of Qwen-Audio-Agent

• Programming CollaborationDevelopers can write code while using voice commands to instruct the Agent to modify files, run tests, check for errors, and interrupt at any time to supplement requirements.
• project managementYou can continuously query the progress of multiple projects, assign tasks, and obtain execution results via voice without switching chat windows.
• Document processingVoice commands drive the Agent to generate, modify, and translate documents, with real-time content confirmation and iterative optimization.
• Intelligent Customer ServiceEnterprises deploy it as a private voice front-end, connecting internal business agents to provide natural and smooth customer interaction.
• AccessibilityIt provides a voice-controlled entry point for users who have difficulty typing, enabling them to perform complex system operations through an Agent.