AB
AB
AiBoss

All Categories

project

MiniCPM-SALA - Wallfacer's open-source 9B edge model

MiniCPM-SALA is a 9B edge-side large model from Wallfacer Intelligence, employing an innovative SALA (Sparse-Linear Hybrid Attention) architecture. 75% of its attention is linear, handling efficient global modeling, while 25% is sparse, precisely capturing key information...

What is MiniCPM-SALA?

MiniCPM-SALA is a 9B edge-side large-scale model open-sourced by Wallfacer Intelligence. It adopts an innovative SALA (Sparse-Linear Hybrid Attention) architecture, with 75% linear attention responsible for efficient global modeling and 25% sparse attention for accurate capture of key information. Hybrid positional encoding (HyPE) is introduced to achieve seamless switching between long and short texts. The model achieves million-level context inference for the first time on consumer-grade graphics cards (such as the RTX 5090), with low memory usage and fast inference speed, providing a feasible path for edge agents to deploy long text capabilities.

Main functions of MiniCPM-SALA

  • Million-level long text processingSupports 1M+ tokens context, and for the first time runs a million-length inference on an RTX 5090 consumer graphics card.
  • High-efficiency reasoning accelerationAchieve a 3.5x speedup in inference on 256K sequences compared to dense models of the same size.
  • Low video memory usageBy optimizing the KV Cache, it can still run stably without OOM (Out of Memory) even with a length of 512K to 1M.
  • Long and short text compatibilityThe model maintains comparable generality to Qwen3-8B for short texts, and has a significant advantage for long texts.
  • Edge deployment optimizationDesigned specifically for terminal devices such as mobile phones, cars, and robots, it enables localized long-context agents.

MiniCPM-SALA Technical Principles

  • SALA Hybrid Attention ArchitectureIt combines 75% linear attention (Lightning Attention) with 25% sparse attention (InfLLM v2) in a hierarchical manner. The former undertakes global modeling with O(N) complexity, while the latter calculates key local information on demand, achieving a balance between efficiency and accuracy.
  • HyPE Hybrid Position EncodingThe linear layer retains RoPE to maintain short text performance, while the sparse layer uses NoPE to decouple KV-Cache from location, avoid long-distance decay, and support efficient retrieval of millions of contexts.
  • HALO low-cost migrationBy employing four steps—parameter transformation, hidden state alignment, layer selection, and knowledge distillation—the full attention model is transferred to a hybrid architecture, reducing the training budget to 25% of that required for de novo pre-training.

MiniCPM-SALA project address

  • GitHub repositoryhttps://github.com/openbmb/minicpm
  • HuggingFace model libraryhttps://huggingface.co/openbmb/MiniCPM-SALA

Application scenarios of MiniCPM-SALA

  • Personal intelligent assistantIt continuously retains millions of chat logs and personal preferences on local devices, creating a truly "understanding" and private personal assistant.
  • End-side knowledge baseBased on an offline document library of millions of words, it provides professional Q&A services to meet the localization needs of enterprises for sensitive data.
  • Code Development AssistantIt enables you to understand the entire code repository context at once, facilitating cross-file debugging and complex refactoring tasks.
  • In-vehicle intelligent systemIt integrates vehicle manuals, navigation history, and user habits to provide personalized driving services in an offline in-vehicle environment.
  • Scientific literature analysisThe model can quickly process massive amounts of academic papers and extract cross-document correlation information to assist in review writing.
Back