MiniCPM-Robot - Wallfacer Intelligence Open Source Embodied Intelligence VLA Model Series
MiniCPM-Robot is a lightweight embodied intelligence (VLA) model series from Wallfacer Intelligence, including the MiniCPM-RobotManip general operation model with 1.5B parameters and the MiniCPM-RobotTrack tracking and navigation model with 0.9B parameters...
What is MiniCPM-Robot?
MiniCPM-Robot is a lightweight embodied intelligence (VLA) model series open-sourced by Wallfacer Intelligence. It includes the MiniCPM-RobotManip general operation model with 1.5B parameters and the MiniCPM-RobotTrack tracking and navigation model with 0.9B parameters, both supported by the PhyAI inference framework. The models employ visual token compression and streaming contextual memory technology, supporting long-range precision manipulation by robotic arms and indoor/outdoor target tracking for robot dogs. They can run locally on the edge in weak network environments, with inference latency as low as 120ms, placing them among the top performers in mainstream benchmarks.
Main functions of MiniCPM-Robot
-
MiniCPM-RobotManipIt controls a robotic arm to complete long-range tasks (such as making sandwiches), supports multiple camera inputs, and has context memory capabilities, which can remember task progress and historical actions.
-
MiniCPM-RobotTrackThe robot dog can continuously track a specified target based on natural language commands. It supports single-target tracking, multi-person interference tracking, and fuzzy target tracking, and can run locally in weak network/no network environments.
The technical principle of MiniCPM-Robot
-
Visual Token CompressionFilter out redundant visual information that is irrelevant to the task, reducing the amount of computation required for a single inference.
-
Streaming context memory: Reuse previously processed historical information; new screens only require incremental calculations, avoiding a surge in computational load for long tasks.
-
Self-evolving data pipelineActively expose the model's weaknesses in both simulation and real-world environments, collect error scenarios, and conduct expert correction and closed-loop training.
Follow us on WeChat and reply with "open source",join inAI open source project discussion group
How to use MiniCPM-Robot
- Environmental preparation：
-
Clone the GitHub repository and install the dependencies.
-
Hardware required: RobotManip robotic arm or Yushu Go2 Edu robot dog (RobotTrack).
-
- Model loadingDownload the corresponding model weights from Hugging Face. Load the model using the PhyAI framework and complete quantization and operator optimization.
- Camera access: Configure a wrist camera and a main-view camera (RobotManip) or the original camera of the robot dog (RobotTrack).
- Command input: Input natural language commands (such as "follow the person in black" or "make a sandwich").
- One-click startThe system runs open-source scripts, receives visual and verbal inputs, outputs robot action commands, and executes them.
MiniCPM-Robot's core advantages
-
Minimal parameters, top-tier performanceThe 1.5B parameter is comparable to larger models such as π0.5 in evaluations such as LIBERO, Calvin, and RoboTwin2; the RMBeen context memory score is about 53 points, far exceeding the approximately 10 points of π0.5.
-
Inference delay halvedThe decision delay per frame on H100 is approximately 120ms, which is close to half of π0.5 (234ms).
-
Pure end-side operationThe robot dog can stably track at frequencies above 5Hz using local computing power and can continue to work even when the network is disconnected.
MiniCPM-Robot project address
- Project official website：https://github.com/OpenBMB/MiniCPM-Robot
- HuggingFace model library：
- https://huggingface.co/openbmb/MiniCPM-RobotManip
- https://huggingface.co/openbmb/MiniCPM-RobotTrack
MiniCPM-Robot Competitive Product Comparison
| Dimension | MiniCPM-RobotManip | π0.5 |
|---|---|---|
| Parameters | 1.5B | 3B |
| open source | ||
| LIBERO | 97.5 | 96.9 |
| Calvin | 4.1 | 4.1 |
| RMBench | 53.5 | ~10 |
| Single frame latency | ~120ms | ~234ms |
| Contextual memory | Streaming multiplexing history information | Primarily based on single-frame judgment |
| End-side deployment | Supports local operation | Typically requires cloud computing power. |
Application scenarios of MiniCPM-Robot
-
Catering preparationThe dual-arm robotic arms complete long-distance sandwich making tasks such as picking up bread, adding lettuce, ham, and eggs according to instructions.
-
Exhibition Hall GuideThe robot can perform offline explanations, free-flowing questions and answers, and route planning based on a local knowledge base in an offline environment.
-
Park InspectionThe robot dog can identify illegally parked vehicles, road abnormalities, and problems with public facilities, and sensitive images can be processed locally without going to the cloud.
-
Warehousing and LogisticsThe robot performs long-range operations such as shelf picking and material handling, and automatically retryes and recovers when it fails.
-
Indoor and outdoor followingThe robot dog continuously tracks the target person in weak network environments such as elevators and underground parking lots based on natural language commands.