ERNIE-4.5-21B-A3B-Thinking - A Thinking Model Launched by Baidu

What is ERNIE-4.5-21B-A3B-Thinking?

ERNIE-4.5-21B-A3B-Thinking is a large-scale language model launched by Baidu, focusing on reasoning tasks. It adopts a hybrid expert (MoE) architecture with a total of 21 billion parameters, each token activating 3 billion parameters, and supports a long context window of 128K, making it suitable for complex reasoning tasks. The model builds its core language backbone through text pre-training. In the post-training phase after reasoning enhancement, it utilizes techniques such as supervised fine-tuning (SFT) and progressive reinforcement learning (PRL) to significantly improve its logical reasoning, mathematical computation, and scientific problem-solving capabilities. It supports efficient tool calls and can be integrated with vLLM, Transformers 4.54+, and FastDeploy, making it suitable for scenarios such as program synthesis, symbolic reasoning, and multi-agent workflows.

Main functions of ERNIE-4.5-21B-A3B-Thinking

• Strong reasoning abilityERNIE-4.5-21B-A3B-Thinking excels in areas requiring reasoning skills, such as logical reasoning, mathematical calculation, and scientific problem solving. It can handle complex reasoning tasks and provide users with accurate answers.
• Efficient tool callingThe model supports structured tools and function calls, and can be integrated with vLLM, Transformers 4.54+, and FastDeploy to achieve more efficient task execution and feature expansion.
• Long context understandingIt features a 128K context window, enabling it to understand and process long text information. It is suitable for complex reasoning tasks that require long context, such as long document analysis and multi-step reasoning.
• Multi-domain applicationsIt is widely used in scenarios such as program synthesis, symbolic reasoning, and multi-agent workflow, providing solutions for complex tasks in different fields and meeting diverse business needs.
• Open source and ease of useIt is open source under the Apache-2.0 license and can be used on platforms such as Hugging Face, making it convenient for developers to conduct research and commercial deployment, and lowering the barrier to entry.

The technical principles of ERNIE-4.5-21B-A3B-Thinking

• Hybrid expert architectureERNIE-4.5-21B-A3B-Thinking adopts a hybrid expert (MoE) architecture, which divides the model parameters into multiple expert modules. Each input token only activates a portion of the expert modules, which significantly improves computational efficiency while maintaining model performance.
• Long context windowThe model supports 128K context windows and can handle long text inputs, which is crucial for tasks that require long context understanding, such as complex reasoning and long document analysis.
• Reasoning Enhancement TrainingBy using techniques such as Supervised Fine-Tuning (SFT) and Progressive Reinforcement Learning (PRL), the model is trained with specialized reasoning abilities, enabling it to perform well in tasks such as logical reasoning, mathematical calculation, and scientific problem solving.
• Activation mechanismThe model is designed with an efficient activation mechanism, where each token activates 3B parameters, ensuring efficient operation even with large-scale parameters, while maintaining the model's flexibility and adaptability.

Project address for ERNIE-4.5-21B-A3B-Thinking

• HuggingFace model libraryhttps://huggingface.co/baidu/ERNIE-4.5-21B-A3B-Thinking

Application scenarios of ERNIE-4.5-21B-A3B-Thinking

• Complex reasoning tasksIt is suitable for scenarios that require deep thinking and reasoning, such as logical reasoning, mathematical calculation, and scientific problem solving, and provides accurate analysis and solutions.
• Code generation and optimizationIt can generate and optimize code, helping developers improve programming efficiency, and is suitable for tasks such as program synthesis and symbolic reasoning.
• Multi-agent workflowIt supports multi-agent collaboration and can be used to build complex automated workflows to improve task execution efficiency.
• Long text analysisWith its long context window, it can handle tasks such as long document analysis and complex text reasoning, making it suitable for scenarios such as academic research and business report analysis.
• Tool Invocation and IntegrationIt supports structured tool and function calls, can be integrated with multiple platforms and tools, expands application scenarios, and meets diverse business needs.