Pangu Large Model 5.5 - Huawei's New Generation AI Large Model

What is the Pangu Large Model 5.5?

Pangu Big Model 5.5 is Huawei's latest generation of large-scale artificial intelligence models, released at its Developer Conference (HDC 2025). The model emphasizes "not writing poetry, but doing things," focusing on solving real-world industry problems and driving the intelligent upgrade of various industries. Pangu Big Model 5.5 comprises five fundamental models, respectively targeting the fields of Natural Language Processing (NLP), multimodal computing, prediction, scientific computing, and computer vision (CV).

Pangu Ultra MoE is a deep thinking model with 718 billion parameters, built on the Ascend full-stack hardware and software platform, possessing core capabilities such as efficient long sequence processing, low illusion, and deep research. Pangu Pro MoE is a 72B A16B model that performs exceptionally well in agent tasks, comparable to the 671 billion parameters of DeepSeek-R1. Pangu Large Model 5.5 introduces an adaptive fast and slow thinking fusion technology to improve inference efficiency and provides support for training autonomous driving and embodied intelligent robots through a multimodal world model.

Main features of the Pangu large model 5.5

• Natural Language Processing (NLP)
  • High-efficiency long sequence processing: Through Adaptive SWA and ESA technologies, it can easily handle contexts with a length of 1 million tokens.
  • Low hallucinationInnovative approaches, such as knowledge boundary determination and structured thinking verification, are adopted to improve the accuracy of model reasoning.
  • Fusion of fast and slow thinkingThe adaptive fast and slow thinking technology automatically switches thinking modes according to the difficulty of the question, providing quick responses to simple questions and in-depth thinking for complex questions, improving reasoning efficiency by 8 times.
  • In-depth research capabilitiesPangu DeepDiver excels in applications such as web search and common sense Q&A through long-chain puzzle synthesis and progressive reward mechanism. It can complete more than 10 hops of complex Q&A within 5 minutes and generate professional research reports of more than 10,000 words.
• Multimodal
  • World ModelThis technology constructs a digital-physical space for training intelligent driving and embodied intelligent robots, enabling continuous optimization and iteration. For example, in the field of intelligent driving, it can generate a large amount of training data without relying on high-cost road surveys.
• predict
  • triplet transformer architectureThis method unifies triple encoding and pre-training of data from different industries to improve prediction accuracy and generalization across industries and scenarios.
• Scientific computing
  • AI ensemble forecastFor example, the Shenzhen Meteorological Bureau's "Zhiji" big model, which is an upgrade of the Pangu big model, has for the first time achieved AI ensemble forecasting, which can more intuitively reflect the evolutionary possibilities of weather systems.
• Computer Vision (CV)
  • 30 billion parameter visual large modelIt supports multi-dimensional general vision perception, analysis and decision-making, builds a general vision fault sample library that is scarce in industrial scenarios, and improves the identifiable types and accuracy of business scenarios.

Pangu Large Model 5.5 Model Introduction

• Pangu Ultra MoE
  • Ultra-large scale and sparse activationIt has 718 billion parameters, uses 256 routing experts, activates 8 experts with each token, and has an activation volume of 3.9 billion, featuring ultra-large scale and high sparsity.
  • Advanced architecture designThe system introduces the MLA (Multi-head Latent Attention) mechanism to effectively compress the KV cache space and alleviate the memory bandwidth bottleneck during the inference phase. Simultaneously, it employs MTP (Multi-Token Parallelism) multi-head expansion, extending from a single-head MTP training to a multi-head structure to achieve multi-token speculative inference and accelerate the overall inference process.
  • Stable training techniquesWe proposed the Depth-Scaled Sandwich-Norm (DSSN) stable architecture and the TinyInit small initialization method, which solved the stability problem in the training process of ultra-large-scale MoE models and achieved long-term stable training on more than 18TB of data.
  • High-efficiency load optimizationThe EP group loss load optimization method is adopted to ensure good load balance among experts while improving their domain specialization capabilities.
  • Training strategy optimizationThe dropless training strategy is used to avoid the inconsistency between drop and pad training and inference, thus improving training data efficiency. Furthermore, an iterative hard example mining and multi-ability balancing reward function, referencing the GRPO algorithm, are employed to improve the model's training efficiency and final inference performance.
• Pangu Pro MoE
  • Grouped Hybrid Expert ArchitectureThe innovative MoGE (Modulated Grouped Expert Model) model is proposed. Experts are grouped during the expert selection phase, and tokens are constrained to activate an equal number of experts in each group, thereby achieving expert load balancing and significantly improving the deployment efficiency of the model on the Ascend platform.
  • High-performance inferenceOn the Ascend 300I Duo platform, single-card throughput can reach 201 tokens/s, which can be further improved to 321 tokens/s by introducing MTP decoding and multi-token optimization. On the Ascend 800I A2 platform, millisecond-level response can be achieved in low-concurrency scenarios, and single-card throughput can reach 1148 tokens/s under high-concurrency conditions, which can be improved to 1528 tokens/s after optimization, significantly outperforming dense models of the same scale.
  • Excellent reasoning abilityIt demonstrates outstanding performance in cross-language and multi-domain benchmark tests, covering general English reasoning, reading comprehension, common sense reasoning, code generation and bilingual (Chinese and English) math problems in logical reasoning, as well as knowledge-based question answering and reading comprehension in Chinese, comprehensively verifying the model's versatility and domain adaptability in complex cognitive tasks.
  • Hardware affinity optimizationSystem optimization is performed on the Ascend 300I Duo and 800I A2 platforms, deeply integrating the parallel computing characteristics of the Ascend hardware acceleration architecture with operator-level compilation optimization technology to achieve full-stack innovation from algorithm design to system deployment.
• Pangu Embedding
  • Fusion of fast and slow thinkingIt adopts a dual-system cognitive architecture, integrating "fast thinking" and "slow thinking" dual reasoning modes. Through a two-stage training framework, the first stage builds a basic inference engine through iterative distillation and a multi-source dynamic reward system (MARS); the second stage endows the model with the ability to think fast and slow, and can automatically switch modes according to the difficulty of the task, achieving a dynamic balance between reasoning efficiency and depth.
  • High-efficiency training strategiesWe propose an SFT scheme based on model-aware iterative distillation, which dynamically selects data samples that match the current capabilities of the model for training, and retains early knowledge through model merging strategies during the training process to continuously improve performance.
  • Industry vertical capability expansionBy incorporating long-term thinking data from specific domains for further training, the model's capabilities on specialized tasks can be significantly improved. For example, in the legal field, after training with legal corpora, the average accuracy on the LawBench benchmark test reached 54.59%.
  • Adaptive mode switchingThe model can automatically adjust the inference depth based on the complexity of the task. It quickly outputs answers for simple questions and performs in-depth analysis for complex questions to ensure the accuracy of the output.
• Pangu DeepDiver
  • Complex task processingFor in-depth research scenarios, such as scientific assistants, personalized education, and complex industry report surveys, it can complete more than 10 complex question-and-answer sessions and generate professional research reports of more than 10,000 words.
  • Efficient information acquisitionBy constructing a large amount of synthetic interactive data and using a progressive reward strategy for reinforcement learning training, it performs excellently in open-domain information acquisition, and can complete complex tasks and generate high-quality research reports within 5 minutes.
  • Enhanced high-level abilitiesIt significantly enhances the Pangu Big Model's higher-order capabilities such as autonomous planning, exploration, and reflection, making it perform better in handling complex tasks.
• Pangu Prediction ModelIt adopts the industry's first triplet transformer unified pre-training architecture, which encodes data from different industries into triplets in a unified manner, and processes and pre-trains them efficiently within the same framework, improving the accuracy of large prediction models and their generalization across industries and scenarios.
• Pangu Scientific Computing ModelHuawei Cloud continues to expand the integration of the Pangu scientific computing big data model with more scientific application fields. For example, the Shenzhen Meteorological Bureau has further upgraded the "Zhiji" big data model based on Pangu, achieving AI ensemble forecasting for the first time. This can more intuitively reflect the evolutionary possibilities of weather systems and reduce the error of single forecast models.
• Pangu Computer Vision (CV) Large ModelHuawei Cloud has released a new MoE architecture-based visual model with 30 billion parameters, currently the industry's largest visual model. It fully supports multi-dimensional, general-purpose visual perception, analysis, and decision-making, including image, infrared, laser point cloud, spectral, and radar data. By generating models across dimensions, it constructs a rare general-purpose visual fault sample library for industrial scenarios such as oil and gas, transportation, and coal mining, improving the identifiable types and accuracy of faults in business scenarios.
• Pangu Multimodal Large ModelA brand-new world model based on the Pangu multimodal model has been released, which can construct the digital physical space required for the training of intelligent driving and embodied intelligent robots, and achieve continuous optimization and iteration.

The project address for Pangu Large Model 5.5

• Pangu Ultra MoE technical paper: https://arxiv.org/pdf/2505.04519

  Pangu Pro MoE project address: https://gitcode.com/ascend-tribe/pangu-pro-moe

  Pangu Embedding technical paper: https://arxiv.org/pdf/2505.22375

  Pangu DeepDiver technical paper: https://arxiv.org/pdf/2505.24332

Application scenarios of the Pangu large model 5.5

• Intelligent drivingThe Pangu multimodal large model can generate a large amount of training data for intelligent driving without relying on high-cost road surveys.
• Embossed intelligent robotThe Pangu world model can construct the necessary digital physical space for training embodied intelligent robots, enabling continuous optimization and iteration.
• Weather forecastThe Shenzhen Meteorological Bureau's "Zhiji" big model, an upgrade of the Pangu scientific computing big model, has for the first time achieved AI ensemble forecasting, which can more intuitively reflect the evolution of weather systems and reduce the error of single forecast models.
• Industrial ScenePangu CV's large model generates models across dimensions, constructing a rare general visual fault sample library for industrial scenarios such as oil and gas, transportation, and coal mining, which greatly improves the identifiable types and accuracy of business scenarios.