Xiaomi MiMo - Xiaomi's first open-source large-scale inference model

What is Xiaomi MiMo?

Xiaomi MiMo is Xiaomi's first open-source large-scale reasoning model, supporting improved performance in complex reasoning tasks. Based on combined pre-training and post-training, the model mines a large amount of rich reasoning corpus and employs innovative reinforcement learning algorithms to significantly enhance mathematical reasoning and code generation capabilities. With only 7B parameters, MiMo outperforms larger-scale models such as OpenAI's o1-mini and Alibaba Qwen's QwQ-32B-Preview on public benchmark datasets. Xiaomi MiMo includes four model versions: the pre-trained model MiMo-7B-Base, the supervised fine-tuning model MiMo-7B-SFT, and the reinforcement learning models MiMo-7B-RL and MiMo-7B-RL-Zero, all of which have been open-sourced on HuggingFace, providing developers with powerful reasoning tools.

Xiaomi's newly released open-source large language model, MiMo-V2-Flash, employs a 5:1 ratio of alternating sliding window attention (window size 128) and global attention, reducing key-value caching by nearly 6 times while balancing performance and efficiency for long texts. It supports parallel prediction of multiple tokens, with an average accepted length of 2.8-3.6, and inference speed improved by 2-2.6 times, particularly optimizing encoding task efficiency. In multiple benchmark tests, it rivals leading open-source models such as DeepSeek-V3.2 and Kimi-K2.

Main functions of Xiaomi MiMo

• Strong mathematical reasoning abilityIt solves complex mathematical problems and provides accurate reasoning paths and answers.
• Highly efficient code generation capabilitiesGenerates high-quality code suitable for a variety of programming tasks.
• Optimized inference performanceBased on the synergy between pre-training and post-training to enhance inference capabilities, it surpasses larger-scale models with a parameter scale of 7B, demonstrating highly efficient inference performance.

The technical principles of Xiaomi MiMo

• Pre-training phaseThe focus was on mining rich reasoning corpora, synthesizing approximately 200B of reasoning data to ensure the model had encountered more reasoning patterns. Based on three-stage training, the training difficulty was gradually increased, with a total training volume of 25T tokens, allowing the model to progressively improve its capabilities in tasks of varying difficulty.
• Post-training phase:
  • Reinforcement learning algorithmsIntroducing the Test Difficulty Driven Reward algorithm to alleviate the reward sparsity problem in difficult algorithm problems and improve the performance of models in complex tasks.
  • Data resampling strategyIntroducing the Easy Data Re-Sampling strategy to stabilize the reinforcement learning (RL) training process.
  • High-efficiency training frameworkDesign a Seamless Rollout system to accelerate RL training (2.29x) and validation (1.96x), improving training efficiency.
• Model architecture optimization: Optimize the model architecture for inference tasks to ensure efficient inference capabilities with a limited number of parameters.

Xiaomi MiMo project address

• GitHub repository:https://github.com/XiaomiMiMo
• HuggingFace model library:https://huggingface.co/XiaomiMiMo
• Technical Papers:https://github.com/XiaomiMiMo/MiMo/blob/main/MiMo-7B-Technical-Report.pdf

Application scenarios of Xiaomi MiMo

• EducationIt assists in solving math problems and learning programming by providing solution steps and code examples.
• Scientific research and academic: Assists in logical reasoning and algorithm development, and helps verify hypotheses and design experiments.
• Software developmentGenerate and optimize code, and assist in code debugging and problem solving.
• Intelligent Customer Service: To answer complex questions and improve the efficiency of the question-and-answer system.
• Games and EntertainmentIt provides strategic advice and solutions to brain teasers, increasing the fun of the game.