AB
AB
AiBoss

All Categories

project

Qwen3-Next - Alibaba Tongyi's Open Source Hybrid Architecture Model

Qwen3-Next is a new generation of hybrid architecture model open sourced by Alibaba Tongyi, which includes two versions: the instruction version (Qwen3-Next-80B-A3B-Instruct) and the thinking version (Qwen3-Next-80B-A3B-Thinking).

What is Qwen3-Next?

Qwen3-Next is a new generation of hybrid architecture large-scale model open-sourced by Alibaba Tongyi, including two versions: an instruction version (Qwen3-Next-80B-A3B-Instruct) and a thinking version (Qwen3-Next-80B-A3B-Thinking). The model utilizes a world-first hybrid architecture of Gated DeltaNet and Gated Attention to achieve faster processing speed, stronger performance, and lower resource consumption. The model has a total of 80B parameters, activating only about 3B parameters per inference. A pre-training acceleration mechanism significantly improves the speed of long-text generation. Qwen3-Next is suitable for scenarios requiring long-context inference, high concurrency, and low latency. The model is now available on Alibaba Cloud's Bailian platform, supporting API calls, and can be experienced online via the QwenChat web version.

Main functions of Qwen3-Next

  • Instruction comprehension and executionThe specially optimized Instruct version excels at understanding and executing user commands.
  • Deep thinking and reasoningThinking supports multi-step reasoning and in-depth thinking.
  • Long text processingThe model can process and understand long text inputs of more than 32K characters.
  • High-efficiency reasoningBased on a hybrid engine, combining speed and precision, it enables fast and accurate reasoning.
  • Save resourcesWhile maintaining the capabilities of large models, computational resources are saved by activating some parameters.

The technical principles of Qwen3-Next

  • Hybrid architectureIt combines 75% Gated DeltaNet and 25% Gated Attention to balance speed and accuracy.
    • Gated DeltaNetUsed for processing long texts, providing fast inference speed and linearly increasing memory footprint.
    • Gated Attention: Responsible for accurately recalling key information and ensuring that the model does not lose focus when processing long texts.
  • Extremely cost-effective architectureThe model has a total of 80B parameters, and only about 3B parameters are activated during each inference, which significantly reduces the computational cost.
  • Pre-training acceleration mechanismBy using native MTP (Multi-Token Prediction) acceleration technology, the model can predict multiple tokens at once during the pre-training stage, reducing the number of inference steps and improving throughput when generating long texts.
  • Expert Systems (MoE)The model contains 512 experts. Each time a request is processed, the 10 most relevant experts and 1 shared expert are dynamically selected to work, thereby achieving load balancing and resource optimization.

Qwen3-Next project address

  • HuggingFace model library: https://huggingface.co/collections/Qwen/qwen3-next-68c25fd6838e585db8eeea9d

Qwen3-Next performance

  • Instruct Model RepresentationIn multiple benchmark tests, the Qwen3-Next-80B-A3B-Instruct model's instruction capabilities are comparable to the 235B flagship model, and it performs better in handling long texts.
  • Thinking Model RepresentationThe Qwen3-Next-80B-A3B-Thinking model surpasses the Gemini Flash in inference capabilities, with some metrics even approaching those of the flagship 235B model.

How to use Qwen3-Next

  • Alibaba Cloud Hundred RefinementsVisit the Alibaba Cloud Bailian official website, search for Qwen3-Next, and call the model according to the provided API reference.
  • QwenChat Web VersionYou can directly visit the QwenChat official website to interact with the model online.
  • Hugging FaceOn Hugging Face, find the Qwen3-Next model collection and use and experience the models.

Application Scenarios of Qwen3-Next

  • Intelligent Customer ServiceLeveraging the ability to understand and execute model instructions, we provide automated customer service and support.
  • Content generationIt can automatically generate articles, stories, or other text content in fields such as news, blogs, and social media.
  • Data AnalysisData analysts analyze large amounts of text data to extract key information, supporting market research and business intelligence.
  • Educational SupportIn the field of education, it assists students in learning by providing personalized teaching content and tutoring.
  • Legal ConsultationLegal professionals analyze legal documents and provide legal advice and research support.
Back