project
Qwen3.8-Flash - A multimodal MoE model launched by Alitongyi
Qwen3.8-Flash is a multimodal MoE model launched by Alibaba Cloud Tongyi Qianwen. It has a total of 125B parameters, with only 6B activated per token, and natively supports 262K contexts.
What is Qwen3.8-Flash?
Qwen3.8-Flash is a multimodal MoE model launched by Alibaba Cloud Tongyi Qianwen. It has a total of 125B parameters, with only 6B activated per token, and natively supports 262K contexts. The model adopts four major architectural innovations: GDN+QSA hybrid attention, gated residuals, N-gram embeddings, and Muon optimizers. Its training cost is only 1/9 of that of Qwen3.7-Plus, making it more capable in coding and office tasks.
Main functions of Qwen3.8-Flash
-
Long context processingIt natively supports 262K tokens and can be expanded to 1M, making it suitable for analyzing long documents and code bases.
-
Code-based intelligent agentsSupports end-to-end software engineering tasks, including code generation, debugging, and multi-language development.
-
Office AutomationIt enables the execution of long-term office processes and professional work tasks, and integrates the ability to call up tools.
-
Multimodal understandingUnderstanding long videos, scientific charts, visual math problems, and real-world scenarios.
-
Device and Interface ControlControl web pages, Android devices, and desktop operating systems through visual perception.
-
Reasoning and QuestionsIt covers scientific reasoning, competitive programming, instruction following, and multidisciplinary integrated reasoning.
The technical principles of Qwen3.8-Flash
- GDN + QSA Hybrid AttentionThe model incorporates three Gated DeltaNet layers and one Qwen Sparse Attention layer in every four layers to achieve a hybrid attention mechanism of "efficient memory + accurate search". In a 1M context, the prefill throughput is 8.6 times that of the previous generation.
- Gated Residual（GR）It expands the traditional single residual stream into four parallel branches, controls the read and write ratio of information at each layer through dynamic gating, preserves long-distance information channels, suppresses activation outliers, and supports FP8 storage to significantly reduce memory access overhead.
- N-gram EmbeddingIn addition to regular word embedding, a 51-byte N-gram embedding layer is introduced, which captures phrase collocation patterns by combining local context lookup tables. This layer can be offloaded to host memory and overlapped with computation through asynchronous prefetching, with almost no increase in token-level computation.
- Muon OptimizerMuon orthogonalization optimization is used for the main linear mapping weights, while AdamW is retained for Embedding and Router; after refitting the Scaling Law, larger learning rates and batches are supported, and the ineffective Batch Size Warmup strategy is removed.
Follow us on WeChat and reply with "open source",join inAI open source project discussion group
How to use Qwen3.8-Flash
-
Open source weight deploymentDownload the Qwen3.8-Flash-Next weights from Hugging Face and deploy them for inference locally or in the cloud.
-
API service callBy calling the official API through the Qianwen AI platform, inputting 1 yuan per million tokens, the output is 3 yuan per million tokens.
-
Qianwen Office UseExperience it directly in "Qianwen Office" for long document processing, code generation, and office automation tasks.
-
Review technical reportsVisit the technical reports in the GitHub repository to learn about the architecture details and training methods for in-depth customization.
Qwen3.8-Flash's core advantages
-
Ultimate cost efficiencyTraining overhead is only 1/9 of Qwen3.7-Plus, API pricing is 1 yuan per million tokens, achieving flagship-level performance with extremely low computing power cost.
-
Extremely long context leadThe native 262K can be scaled up to 1M tokens. In a 1M context, the prefill throughput is 8.6 times that of the previous generation, achieving both speed and accuracy in long sequence processing.
-
Agent and coding capabilities are outstandingIt significantly outperforms similar models in multiple benchmarks, including code intelligence, software engineering, and long-term office tasks, and is competent in complex production-level development work.
-
Next-generation architectural innovationIt adopts a brand-new design with GDN+QSA hybrid attention, gated residuals and N-gram embedding, and is an early verification version of the Qwen4 series architecture.
-
Multimodal control capabilityIt supports long video understanding, visual web page development, and Android and desktop device control, enabling true multimodal intelligent agent interaction.
-
Open source and engineering friendlyFlash-Next weights are now open source. GR supports FP8 and N-gram embedding, which can be offloaded to Host Memory, reducing deployment barriers and GPU memory usage.
Qwen3.8-Flash project address
- Project official website：https://qwen.ai/blog?id=qwen3.8-flash-next
- HuggingFace model library：https://huggingface.co/Qwen/Qwen3.8-Flash-Next?spm=a2ty_o06.30285417.0.0.1d73c921FsyOPe&file=Qwen3.8-Flash-Next
- Technical Papers：https://github.com/QwenLM/Qwen3.8-Flash-Next/blob/main/tech_report.pdf
Comparison of Qwen3.8-Flash with similar competing products
| Comparison Dimensions | Qwen3.8-Flash-Next | DeepSeek-V4-Flash | Claude-Opus-4.6 |
|---|---|---|---|
| Total number of parameters | 125B + 51B N-gram | 284B | — |
| Activation parameters | 6B | 13B | — |
| Agentic Coding (DeepSWE 1.1) | 58.7 | 54.4 | — |
| SWE-bench Pro | 62.5 | 56.0 | 53.4 |
| Multilingual software engineering | 81.0 | — | 77.5 |
| Long-distance office (CoWorkBench) | 73.9 | 45.1 | 68.2 |
| professional work (JobBench) | 55.7 | 41.3 | 36.6 |
| Scientific reasoning (GPQA Diamond) | 91.7 | 90.8 | 91.3 |
| Competitive programming (LiveCodeBench) | 91.9 | 90.6 | 88.8 |
| Instructions followed (IFBench) | 81.3 | 79.2 | 62.5 |
| Multimodal tools (ClawEval-MM) | 64.4 / 60.4 | — | 52.5 / 54.7 |
| Mobile control (AndroidWorld) | 84.5 | — | 62.0 |
| Computer use (OSWorld 2.0) | 19.4 / 52.3 | — | — |
| Visual web development (Vision2Web) | 64.0 | — | — |
| Embodied Intelligence (ERQA) | 72.3 | — | 40.8 |
| Long video comprehension (LVBench) | 76.6 | — | 63.0 |
| Visual Mathematics (MathVision w/ CI) | 95.7 | — | 65.5 |
| Scientific chart analysis (CharXiv w/ CI) | 90.6 | — | 66.0 |
Applications of Qwen3.8-Flash
-
Intelligent analysis of ultra-long documents and knowledge base: Use 1M tokens for ultra-long contexts to achieve in-depth understanding and Q&A of an entire technical manual, legal contract, or large codebase in one go, without the need for segmentation.
-
End-to-end software development and code maintenanceAs an agentic coding agent, it performs tasks such as requirements analysis, code generation, bug fixing, unit test writing, and multi-language project maintenance, covering complex tasks at the SWE-bench level.
-
Enterprise Office Process Automation: Perform cross-platform long-term office tasks, such as automatically organizing meeting minutes, batch processing Excel reports, writing and formatting business documents, and integrating tool calls to achieve end-to-end closed loop.
-
Multimodal content comprehension and educational guidanceIt analyzes long video courses, scientific paper charts, and visual math problems, providing step-by-step explanations and answer verification to support in-depth teaching assistance in the STEM field.
-
Cross-device intelligent agent controlThrough visual perception and reasoning capabilities, it can automatically control Android phones to complete App testing, perform file management in desktop operating systems, or automatically generate deployable web front-ends based on design drafts.