BitCPM-CANN - A large-scale edge model jointly developed by Wallfacer Intelligence and Tsinghua University.

What is BitCPM-CANN?

BitCPM-CANN is a 1.58-bit ternary large-scale model jointly launched by Wallfacer Intelligence, Tsinghua University, and the OpenBMB open-source community. It is China's first 1.58-bit ternary large-scale model trained end-to-end entirely on Huawei's Ascend computing platform. The model adopts a quantization-aware training route, including four sizes from 0.5B to 8B, releasing approximately 6 times the GPU memory advantage during the inference stage. The model capability retention rate reaches 90.1%–97.2%, marking the first time that a domestically produced NPU has a complete low-bit training stack.

Main functions of BitCPM-CANN

• 1.58-bit ternary large model inference: It offers four parameter specifications: 0.5B, 1B, 3B, and 8B, and can run efficiently on edge devices. The 8B model can be easily deployed on current mainstream flagship mobile phones.
• Domestic Ascend full-chain training closed loop: The entire training and inference process is completed natively on the Huawei Ascend platform, without relying on foreign GPUs for pre-training.
• Ultimate memory release: Compared to traditional BF16 precision, it releases about 6 times the GPU memory advantage during the inference stage, and can support far greater model capabilities with the same amount of memory.
• Full-precision comparison and evaluation verification: It achieves a 1:1 performance comparison with the same-sized MiniCPM4 full-precision model in 11 tasks including common sense, reading comprehension, subject knowledge, mathematics and reasoning.
• Open source models and training platforms: All model weights and the low-bit training base based on MindSpeed × Megatron-LM are made available to the public, supporting developers to reproduce and innovate upon them.

BitCPM-CANN Technical Principles

• Quantitative Awareness Training (QAT): The model actively learns to use three-valued weights (-1, 0, +1) to represent knowledge during the initial training phase, and is passively compressed after training is completed, thus fundamentally achieving native growth at extremely low bit depths.
• Maximizing the information density of three-valued weights: By constraining the bit width to 1.58, each bit is forced to maximize its knowledge carrying efficiency, making the weight accuracy dependent on the amount of knowledge carried by each bit, not the bit width.
• MindSpeed × Megatron-LM Training Base: A complete low-bit training infrastructure is built on the Ascend platform based on the backbone framework, including environment adaptation, 32K long sequence support, parallel strategies and fusion operator system.
• End-to-end native adaptation of domestic computing power: From the lowest-level quantization operators and QAT algorithm to the complete parallel strategy and training framework, everything is natively developed and optimized for Huawei Ascend, achieving integrated training and inference.
• Systematic scalable low-bit paradigm: Engineering validation across a full range of sizes from 0.5B to 8B demonstrates that this low-bit training approach possesses cross-scale systematicity, scalability, and engineering reproducibility.

How to use BitCPM-CANN

• Obtaining open-source model weights: Visit HuggingFace to download the full range of model weights: 0.5B, 1B, 3B, and 8B.
• Deploy the training base: Based on the MindSpeed × Megatron-LM low-bit training platform developed by Mianbi Intelligence, environment adaptation, dependency installation, and framework deployment were completed on the Huawei Ascend platform.
• End-side inference execution: Leveraging the model's 6x VRAM advantage, inference can be performed directly on current mainstream flagship smartphones or PC-based devices with 8B or less of VRAM.
• Innovation in secondary training: Based on the QAT training infrastructure and open-source code, we can innovate low-bit training on the Ascend platform by performing secondary fine-tuning, domain adaptation, or larger-scale training.

BitCPM-CANN's core advantages

• Breakthrough in China's domestic computing power closed-loop system: For the first time, end-to-end 1.58-bit training and full-precision comparative evaluation were achieved on Ascend, breaking the stereotype that domestic chips can only run inference.
• Industry-leading performance retention rate: The 3B model has the highest capability retention rate of 97.2%, followed by 8B at 95.7% and 0.5B at 90.1%, demonstrating systematic scalability and engineering reproducibility.
• The training efficiency is significantly better than similar methods: Previously, the GPU version of BitCPM 1B achieved a complete victory with only 1/20 of the training computing power of Microsoft BitNet 2B, proving its efficiency advantage.
• The end ceiling was raised significantly: With a 6x increase in video memory combined with the MoE architecture, it is expected to be able to load ultra-large models with 50B or even 100B parameters into the terminal, bringing edge AI capabilities to a whole new level.

BitCPM-CANN project address

• HuggingFace model libraryhttps://huggingface.co/collections/openbmb/bitcpm-cann

Comparison of BitCPM-CANN with similar products

Application scenarios of BitCPM-CANN

• AI on smartphones: Running larger parameter models with limited memory is expected to enable the running of 60B large models on mobile phones in the future, directly improving the intelligence level of terminals and user experience.
• Complete the domestic chip ecosystem: It provides high-quality model-side supply for edge chips such as Qualcomm Snapdragon 8 Gen 4 that already support 2-bit inference, realizing software and hardware collaboration and ecosystem closed loop.
• Enterprise-level edge deployment: It is suitable for low-memory cost large model inference in PCs, automobiles, IoT devices, etc., to hedge against the supply chain pressure caused by the surge in global HBM memory prices.
• Innovations in Low-Bit Training Research: Provide reusable and scalable QAT public infrastructure for Ascend platform developers, supporting secondary training and fine-tuning innovation based on domestic computing power.