Qwen3 - A new generation of hybrid inference models open sourced by Alibaba Tongyi

What is Qwen3?

Qwen3 is a new generation of large-scale language models launched by Alibaba. Qwen3 supports two working modes: "thinking mode" and "non-thinking mode." In thinking mode, the model reasons step-by-step, providing a final answer after careful consideration, suitable for complex problems. In non-thinking mode, the model offers fast, near-instantaneous responses, suitable for simple problems. Qwen3 supports 119 languages and dialects, a significant improvement over the 29 languages of its predecessor. Qwen3 optimizes encoding and agent capabilities, supports the MCP protocol, and can better integrate with external tools and data sources. Qwen3's dataset size reaches approximately 36 trillion tokens, twice that of Qwen2.5. It employs a four-stage training process, including long thought chain cold start, long thought chain reinforcement learning, thought pattern fusion, and general reinforcement learning. The Qwen3 series of models is open-source under the Apache 2.0 license, and can be downloaded and used commercially by developers, research institutions, and enterprises worldwide.

Alibaba has recently open-sourced two new Qwen3 series models: Qwen3-Embedding and Qwen3-Reranker.

Qwen3-Embedding: In multilingual text representation benchmarks, Qwen3-Embedding demonstrates outstanding performance. The 8-parameter model ranked first on the MTEB Multilingual Leaderboard with a score of 70.58, surpassing many commercial API services, such as Google's Gemini-Embedding. It accepts a single text segment as input and uses the hidden state vector corresponding to the "EOS" label in the last layer of the model as the semantic representation of the input text. It is suitable for scenarios requiring semantic representation of text, such as text classification, clustering, and similarity calculation, and can provide high-quality text features for downstream tasks.

Qwen3-Reranker: In basic relevance retrieval tasks, the 8B model achieved a high score of 69.02 in multilingual retrieval tasks, 77.45 in Chinese retrieval tasks, and 69.76 in English retrieval tasks, significantly outperforming other baseline models. It takes text pairs (e.g., user queries and candidate documents) as input, calculates and outputs the relevance score of the two texts using a single-tower structure. It is used for text retrieval and ranking tasks, such as result ranking in search engines and answer ranking in question-answering systems, improving the relevance and accuracy of search results.

Qwen3's main functions

• Hybrid reasoning modeQwen3 supports two working modes: "thinking mode" and "non-thinking mode".Thinking patternsSuitable for complex problems, the model will provide an answer after step-by-step reasoning;Non-thinking modeIt provides fast, near-instantaneous responses, suitable for simple problems. It allows users to flexibly control the model's inference process based on the complexity of the task, achieving a balance between cost-effectiveness and inference quality.
• Multilingual supportQwen3 supports 119 languages and dialects, including English, French, Chinese (Simplified and Traditional), Cantonese, etc., which greatly expands its international application scope.
• Enhanced Agent CapabilitiesQwen3 optimizes coding and agent capabilities, supports the MCP protocol, and enables efficient interaction with external tools. Combined with the Qwen-Agent framework, it significantly reduces coding complexity and enables efficient agent operations on mobile phones and computers.
• Multiple model configurationsQwen3 offers a variety of model configurations, including two MoE models (Qwen3-235B-A22B and Qwen3-30B-A3B) and six Dense models (Qwen3-32B, Qwen3-14B, Qwen3-8B, Qwen3-4B, Qwen3-1.7B, and Qwen3-0.6B), covering a wide range of scenarios from small devices to large-scale enterprise deployments.

Qwen3's technical principles

• Large-scale pre-trainingQwen3's pre-training dataset comprises approximately 36 trillion tokens, twice that of Qwen2.5, and covers 119 languages and dialects. The pre-training process consists of three stages:
  • Phase 1 (S1)The model was pre-trained on over 30 trillion tokens with a context length of 4K tokens. This stage provided the model with basic language skills and general knowledge.
  • Phase Two (S2)The dataset was improved by increasing the proportion of knowledge-intensive data (such as STEM, programming, and reasoning tasks), and the model was then pre-trained on an additional 5 trillion tokens.
  • Phase ThreeUsing high-quality long context data to extend the context length to 32K tokens ensures that the model can effectively handle longer inputs.
• Optimized post-trainingTo develop a hybrid model that possesses both reasoning and rapid response capabilities, Qwen3 implemented a four-stage training process:
  • Long Chain-of-Thought Cold StartThe model is fine-tuned using diverse long thought chain data, covering a variety of tasks and domains including mathematics, coding, logical reasoning, and STEM problems.
  • Reasoning-based Reinforcement Learning: Use rule-based rewards to enhance the model’s exploration and research capabilities.
  • Thinking Mode FusionThe model is fine-tuned on a combined dataset that includes long thought chain data and commonly used instruction fine-tuning data, integrating non-thinking patterns into the thinking model.
  • General Reinforcement LearningReinforcement learning is applied to more than 20 general domain tasks, including instruction compliance, format compliance, and agent capabilities, to further enhance the model's general capabilities and correct bad behavior.
• Multiple model configurationsQwen3 offers a variety of model configurations, including:
  • MoE model: Qwen3-235B-A22B and Qwen3-30B-A3B.
  • Dense ModelThe series includes Qwen3-32B, Qwen3-14B, Qwen3-8B, Qwen3-4B, Qwen3-1.7B, and Qwen3-0.6B. These models cover a wide range of scenarios, from small devices to large-scale enterprise deployments.
• Performance optimizationQwen3 offers a significant performance boost while drastically reducing deployment costs. For example, a full-power version can be deployed with just four H20 GPUs, and its memory usage is only one-third that of models with similar performance.

Qwen3's project address

• Project official website:https://qwenlm.github.io/blog/qwen3/
• Github repository:
  • Qwen3:https://github.com/QwenLM/Qwen3
  • Qwen3-Embedding:https://github.com/QwenLM/Qwen3-Embedding
• HuggingFace model library:
  • Qwen3:https://huggingface.co/collections/Qwen/qwen3
  • Qwen3-Embedding:https://huggingface.co/collections/Qwen/qwen3-embedding
  • Qwen3-Reranker:https://huggingface.co/collections/Qwen/qwen3-reranker

Qwen3 performance

Qwen3 performed well in multiple benchmark tests, such as:

• AIME25Qwen3 scored 81.5 points, setting a new open-source record.
• LiveCodeBenchQwen3 scored over 70 points, even outperforming Grok3.
• ArenaHardQwen3 surpassed OpenAI-o1 and DeepSeek-R1 with a score of 95.6.

Application scenarios of Qwen3

• Text generationQwen3 can generate coherent, natural-sounding long texts, suitable for tasks such as automated writing, news generation, and blog post creation. It can generate complete articles or stories based on given prompts.
• Machine translationQwen3 supports 119 languages and dialects, performing exceptionally well in multilingual translation tasks. It can handle translation tasks between multiple language pairs, providing high-quality translation results.
• Automatic generation of legal documentsQwen3 can generate legal documents such as contracts, legal opinions, and litigation documents. By fine-tuning the corpus in the legal field, Qwen3 can generate documents that conform to legal regulations and format.
• Technical Documentation WritingQwen3 can generate detailed technical documents, product specifications, user manuals, and more. By fine-tuning its corpus within the technical field, Qwen3 can help developers and technical support teams automate the generation of industry-standard documentation.
• medical fieldQwen3 can be used to generate medical reports, diagnostic suggestions, and more. Through fine-tuning of data such as medical literature and medical records, Qwen3 can generate professional reports that meet medical standards. It can assist doctors in automatically generating case records during the diagnostic process.
• legal fieldQwen3 can generate highly specialized legal documents by fine-tuning legal literature, precedents, and regulations.