AB
AB
AiBoss

All Categories

project

Qwen3-ASR - A series of open-source speech recognition models from Alibaba Tongyi

Qwen3-ASR is a series of open-source speech recognition models from Alibaba Cloud's Tongyi Qianwen team. It includes two ASR models: a 1.7B high-precision version and a 0.6B high-efficiency version, as well as a dedicated Qwen3-ForcedAligner-0.6B forced alignment model.

What is Qwen3-ASR?

Qwen3-ASR is a series of open-source speech recognition models from Alibaba Cloud's Tongyi Qianwen team. It includes two ASR models: a 1.7B high-precision version and a 0.6B high-efficiency version, as well as a dedicated Qwen3-ForcedAligner-0.6B forced alignment model. The models support 52 languages and dialects, integrated streaming and non-streaming inference, and are robust in complex scenarios such as strong noise, fast speech, and singing. The 1.7B model achieves state-of-the-art (SOTA) performance in Chinese, English, and dialect recognition, while the 0.6B model supports 128 concurrent connections with 2000x throughput, processing 5 hours of audio in 10 seconds.

Main functions of Qwen3-ASR

  • Multilingual recognitionIt supports speech recognition and language identification for 30 major languages, enabling unified multilingual modeling.
  • Dialect coverageIt supports the recognition of 22 Chinese dialects, covering Cantonese, Wu dialect, Minnan dialect and local accents from various provinces.
  • Accent matchingIt adapts to the differences in English accents in 16 countries/regions, improving cross-regional recognition capabilities.
  • Dual-mode reasoningSupports integrated streaming and non-streaming inference, and can process up to 20 minutes of audio at a time.
  • Scenario robustIt is robust to complex scenarios and can cope with challenges such as strong noise, low-quality speech, fast speech speed, and the voices of the elderly and children.
  • Singing transcriptionSupports singing with background music and can transcribe entire Chinese and English songs.
  • Timestamp AlignmentIt provides word-level/sentence-level timestamp alignment to meet the needs of subtitle generation and audio editing.

The technical principle of Qwen3-ASR

  • Speech coding layerIt employs an innovative pre-trained AuT speech encoder to extract high-level acoustic representations, replacing traditional Fbank features and enhancing the ability to generalize to noise and accents.
  • Multimodal baseBased on the Qwen3-Omni multimodal large model, it uses its cross-modal understanding capabilities to achieve direct mapping from speech to text, without the need for traditional HMM/GMM pipelines.
  • Training ParadigmA general acoustic semantic space is established through large-scale multilingual pre-training, and then fine-tuned for scenarios such as dialects, singing, and noise, jointly optimizing language recognition and speech recognition tasks.
  • Efficient ReasoningThe 0.6B model uses the vLLM acceleration engine, supports batch inference and asynchronous services, and achieves 2000 times throughput under 128 concurrency; the streaming version uses a block caching mechanism to balance real-time performance and accuracy.
  • Forced alignmentForcedAligner is based on a non-autoregressive LLM architecture and predicts timestamps through parallel decoding, achieving a single-concurrency RTF of 0.0089, which surpasses the accuracy of traditional CTC and WhisperX solutions.

Qwen3-ASR's project address

  • Project official website: https://qwen.ai/blog?id=qwen3asr
  • GitHub repository: https://github.com/QwenLM/Qwen3-ASR
  • HuggingFace model libraryhttps://huggingface.co/collections/Qwen/qwen3-asr
  • Technical Papers: https://github.com/QwenLM/Qwen3-ASR/blob/main/assets/Qwen3_ASR.pdf

Application Scenarios of Qwen3-ASR

  • Smart MeetingThe model can transcribe multi-person meeting content in real time, supports mixed Chinese and English and various dialects, and automatically generates meeting minutes with timestamps.
  • Video subtitlesIt generates accurate subtitles for movies, TV series, short videos, and live streams, and supports the recognition of singing content with background music and the creation of multilingual translation subtitles.
  • Telephone Customer ServiceIt can stably recognize customer speech in noisy and low-quality call environments, and the model supports real-time streaming transcription and keyword extraction.
  • smart speakerIt is compatible with non-standard pronunciations of the elderly and children, supports far-field voice pickup and dialect interaction, and enhances the home voice control experience.
  • Legal Evidence CollectionIt enables high-precision transcription of audio recordings in complex acoustic environments, providing word-level timestamps for use in court proceedings and content verification.
Back