AB
AB
AiBoss

All Categories

project

Xiaomi-Robotics-0 - Xiaomi's open-source VLA robot model

Xiaomi-Robotics-0 is Xiaomi's first open-source VLA (Vision-Language-Motion) large-scale robot model, boasting 4.7 billion parameters. It employs a MoT hybrid architecture, with the Qwen3-VL multimodal model acting as the 'brain' to understand visual and language commands. Di...

What is Xiaomi-Robotics-0?

Xiaomi-Robotics-0 is Xiaomi's first open-source VLA (Vision-Language-Motion) large-scale robot model, boasting 4.7 billion parameters. It employs a MoT hybrid architecture, with the Qwen3-VL multimodal model acting as the "brain" to understand visual and language commands, and the Diffusion Transformer acting as the "cerebellum" to generate high-frequency action blocks. It innovatively introduces asynchronous execution and Λ-shape attention masks to resolve motion stuttering caused by inference latency, achieving real-time smooth control on consumer-grade graphics cards. It sets state-of-the-art (SOTA) records in simulation benchmarks such as LIBERO and CALVIN, and has been successfully applied to real-world dual-arm operation tasks such as block disassembly and towel folding.

Main functions of Xiaomi-Robotics-0

  • Natural Language UnderstandingThe model can parse vague human commands and identify spatial relationships and operational intentions from visual input.
  • Action generation controlThe model can output high-frequency, smooth motion sequences to drive the robot to perform precise physical operations.
  • Real-time asynchronous executionIt supports parallel inference and execution, eliminating latency and stuttering, and ensuring smooth and consistent actions.
  • Dual-arm coordinated operationIt supports the use of both hands to complete complex and long-cycle tasks such as disassembling building blocks and folding towels.
  • Adaptive policy adjustmentThe model can automatically switch action strategies to respond flexibly when the capture fails or the environment changes.
  • Multimodal capability retentionThe model retains general understanding capabilities such as visual question answering and object detection, preventing catastrophic forgetting.

The technical principles of Xiaomi-Robotics-0

  • MoT Hybrid ArchitectureThe Qwen3-VL-4B multimodal model is used as the "brain" to process visual language input, and the Diffusion Transformer is used as the "cerebellum" to generate actions. The total number of parameters is 4.7 billion, which takes into account both general understanding and fine control.
  • Two-stage trainingThe first stage uses the Action Proposal mechanism to allow the VLM to learn the action distribution and align it with the feature space, mixing visual language and robot data to prevent forgetting; the second stage freezes the VLM and trains DiT specifically to recover accurate action sequences from noise through flow matching.
  • Asynchronous execution mechanismThe robot executes the current action block while inferring the next block in parallel. The Clean Action Prefix uses the action of the previous moment as the input condition to ensure the continuity of the trajectory sequence and eliminate the action gap caused by inference delay from the mechanism.
  • Λ-shape attention maskReplaces DiT's causal attention mask, supports adjacent noise tokens to focus on historical actions to achieve a smooth transition, while prohibiting subsequent tokens from accessing the prefix, forcing them to focus on visual signals, avoiding the model from copying inertial actions, and improving the sensitivity to sudden changes in the environment.

Xiaomi-Robotics-0 project address

  • Project official websitehttps://xiaomi-robotics-0.github.io/
  • GitHub repository: https://github.com/XiaomiRobotics/Xiaomi-Robotics-0
  • HuggingFace model libraryhttps://huggingface.co/collections/XiaomiRobotics/xiaomi-robotics-0
  • Technical Papers: https://xiaomi-robotics-0.github.io/assets/paper.pdf

Application scenarios of Xiaomi-Robotics-0

  • Industrial precision assemblyThe model can accurately disassemble complex assemblies consisting of up to 20 building blocks, making it suitable for precision assembly scenarios such as electronic products and automotive parts.
  • Home cleaning servicesThe model can actively shake a towel to expose obscured corners, identify extra items and put them back, making it suitable for home assistance and elderly care scenarios.
  • Logistics warehousing and sortingThe model, with its high-frequency smooth motion generation capability, can adapt to the diverse processing needs of goods of different shapes and materials.
  • Scientific research, education and developmentThe model supports universities and research institutions in conducting research on embodied intelligence algorithms, developing robot control strategies, and providing teaching demonstrations.
  • Business Interactive DisplayThe model can be deployed in scenarios such as showrooms, stores, and press conferences, showcasing low-latency, high-smooth human-computer collaboration capabilities and enhancing the brand's technological image.
Back