MiMo-Embodied - Xiaomi's cross-domain embodied large model

What is MiMo-Embodied?

MiMo-Embodied is the world's first open-source, cross-domain embodied model released by Xiaomi. It integrates tasks from both autonomous driving and embodied intelligence, achieving superior performance in environmental perception, task planning, and spatial understanding. Based on a Visual Language Model (VLM) architecture, the model employs a four-stage training strategy—including embodied intelligence-supervised fine-tuning, autonomous driving-supervised fine-tuning, chained inference fine-tuning, and reinforcement learning fine-tuning—significantly improving its cross-domain generalization capabilities. In the autonomous driving domain, MiMo-Embodied can accurately perceive traffic scenes, predict dynamic target behavior, and generate safe and efficient driving plans. In the embodied intelligence domain, it can understand natural language instructions and complete complex task planning and spatial reasoning. MiMo-Embodied outperforms existing open-source and dedicated models in multiple benchmark tests, demonstrating powerful multimodal interaction capabilities.

Main functions of MiMo-Embodied

• Cross-domain integration capabilitiesMiMo-Embodied is the first model to successfully integrate autonomous driving and embodied intelligence tasks, covering core capabilities such as environmental perception, task planning, and spatial understanding, and is suitable for multimodal interaction in complex dynamic environments.
• Environmental perceptionIn autonomous driving scenarios, the model can accurately understand traffic conditions, including identifying key elements such as traffic signs, vehicles, and pedestrians, and predicting their dynamic behavior, thus providing support for safe driving.
• Task planning and executionIn the field of embodied intelligence, MiMo-Embodied can generate executable action sequences based on natural language instructions to complete complex task planning, such as robot navigation and operation.
• Spatial understanding and reasoningThe model possesses powerful spatial reasoning capabilities, enabling it to understand spatial relationships between objects and support tasks such as navigation, interaction, and scene understanding. It is suitable for path planning in robot operation and autonomous driving.
• Multimodal interactionThrough deep fusion of vision and language, MiMo-Embodied can process image, video, and text input, supporting multimodal tasks such as visual question answering, instruction following, and scene description.
• Reinforcement learning optimizationBy employing reinforcement learning for fine-tuning, the model's decision-making ability and task execution reliability in complex scenarios are improved, ensuring efficient deployment in real-world environments.
• Open source and universalityMiMo-Embodied is completely open source, with its code and models available on Hugging Face. It provides researchers and developers with powerful tools to drive innovation in the fields of embodied intelligence and autonomous driving.

The technical principle of MiMo-Embodied

• Cross-domain integration architectureMiMo-Embodied employs a unified visual language model (VLM) architecture, integrating autonomous driving and embodied intelligence tasks into a single model. It achieves deep fusion of visual input and text understanding through a visual encoder, projector, and large language model (LLM).
• Multi-stage training strategyThe model improves its performance through four stages of training, including embodied intelligence supervised fine-tuning, autonomous driving supervised fine-tuning, chained inference fine-tuning, and reinforcement learning fine-tuning, to ensure generalization ability in different tasks and scenarios.
• Visual input processingThe Vision Transformer (ViT) is used to encode single images, multiple images, and videos, extract visual features, and map them to a latent space aligned with the LLM through a multilayer perceptron (MLP), thereby achieving seamless integration of vision and language.
• Data-driven cross-domain learningIt has constructed a diverse dataset covering general visual language understanding, embodied intelligence, and autonomous driving scenarios, providing rich multimodal supervision signals for the model and supporting learning from basic perception to complex reasoning.
• Reinforcement learning optimizationIn the final stage of training, the Group Relative Policy Optimization (GRPO) algorithm is used for reinforcement learning fine-tuning to optimize the decision quality and reliability of the model for complex tasks and edge scenarios.
• Reasoning and Output GenerationBy leveraging the reasoning capabilities of LLM, visual input is combined with language commands to generate task-related responses and decisions, supporting path planning in autonomous driving and task execution in embodied intelligence.

MiMo-Embodied project address

• Github repositoryhttps://github.com/XiaomiMiMo/MiMo-Embodied
• HuggingFace model libraryhttps://huggingface.co/XiaomiMiMo/MiMo-Embodied-7B
• arXiv technical paperhttps://arxiv.org/pdf/2511.16518

Application scenarios of MiMo-Embodied

• autonomous drivingMiMo-Embodied can handle complex traffic scenarios, perform environmental perception, state prediction, and driving planning, and is suitable for various autonomous driving scenarios such as urban roads and highways, providing decision support for intelligent driving systems.
• Robot navigation and operationIn the field of embodied intelligence, models can perform tasks such as indoor navigation and object manipulation based on natural language instructions, supporting autonomous actions of robots in environments such as homes and industries.
• Visual Question Answering and InteractionSuitable for visual question answering (VQA) tasks, it can understand image or video content and answer related questions, supporting information retrieval and interpretation in human-computer interaction.
• Scene understanding and descriptionThe model can perform semantic understanding and generate descriptions for complex scenes, and is suitable for scene analysis in fields such as security monitoring and intelligent transportation.
• Multimodal task executionIt supports multimodal input, such as images, videos, and text, and can handle cross-modal tasks, such as instruction following and image annotation, making it suitable for intelligent assistants and automation systems.
• Task planning in complex environmentsIn complex environments, MiMo-Embodied can generate multi-step task plans based on instructions, enabling robots to complete complex tasks such as cleaning and cooking.