project
ACE-Ego - An embodied manipulation VLA model jointly developed by DaXiao Robotics and the Chinese University of Hong Kong.
ACE-Ego is a brain-based multi-morphic embodied manipulation VLA model open-sourced by DaXiao Robotics in collaboration with the Chinese University of Hong Kong's MMLab. The model was pre-trained using over 6.0K hours of human first-person perspective video as its core, and incorporates camera-space motion and morphological conditions...
What is ACE-Ego?
ACE-Ego is an open-source, multi-aspect embodied manipulation VLA model developed by Daxiao Robotics in collaboration with MMLab at the Chinese University of Hong Kong. The model is pre-trained using over 6.0K hours of human first-person perspective video, and through camera-space motion and morphological conditional encoding, the same model can be adapted to different robot bodies. It achieved a state-of-the-art performance of 72.8% on the RoboCasa GR1 TableTop benchmark, stably completing complex retail operations such as packing plastic bags and shoe boxes, significantly reducing the deployment cost of multi-aspect embodied robots.
Main functions of ACE-Ego
- One brain, multiple types of adaptationThe same VLA model can be adapted to different robot bodies through morphological conditional encoding, eliminating the need for separate training for each robot.
- Execution of complex operationsSupports long-cycle, high-contact retail operations, such as packing in plastic bags and putting shoes into shoe boxes.
- Bi-arm Coordination ControlAchieved a 90.62% success rate in the high-difficulty bi-arm operation benchmark RoboTwin 2.0, supporting complex two-handed collaboration.
- Open source ecosystemThe model and related resources are open-sourced to the industry, lowering the barrier to entry for embodied intelligence research and development.
The technical principle of ACE-Ego
- Human-centered pre-trainingThe system integrates over 6,000 hours of first-person human video, multi-body robot demonstrations, and simulation data for pre-training. It uses videos of everyday human operations to establish general operational representations and fine-tunes them with a small amount of robot data to achieve skill transfer.
- Camera Space MotionThis method unifies motion prediction to the camera coordinate system, eliminating inconsistencies in motion space caused by differences in robot bodies and camera perspectives, and enabling the model to output unified motion commands across different shapes.
- Morphological condition codingBy encoding robot morphological information, such as joint structure and degree of freedom configuration, the model can understand the physical constraints of different bodies and adapt to various robot hardware within the same model.
- Reliability Awareness TrainingA reliability assessment mechanism is introduced to distinguish between high-confidence and low-confidence action predictions during training, thereby improving the stability and safety of the model in complex contact operations.
Follow us on WeChat and reply with "open source",join inAI open source project discussion group
How to use ACE-Ego
- Environmental preparationClone the official GitHub repository and install the Python dependencies to ensure your system supports PyTorch and CUDA.
- Model DownloadObtain the ACE-Ego pre-trained weight file from open source and load the human-centered pre-trained VLA base model.
- Ontology configurationThe target robot's URDF structure, joint degrees of freedom, and camera parameters are input through the morphological condition coding interface to complete the one-brain-multiple-type adaptation.
- Camera calibrationThe intrinsic and extrinsic parameters of the RGB-D camera mounted on the robot are calibrated to establish a unified camera space motion coordinate system.
- Data fine-tuning (optional)Collect a small amount of robot demonstration data for a specific task, and then perform lightweight fine-tuning by combining it with pre-trained weights from human videos.
- Visual inputInput the robot's real-time camera footage and voice commands, such as "put the shoes into the shoebox," into the model.
- Action reasoningThe model outputs the end effector motion sequence in camera space, including position, pose, and gripper opening/closing degree.
- Mapping executionIt converts camera-space motion commands into joint control signals for the target robot body, driving the hardware to complete the operation.
- Reliability verificationEnable the reliability awareness module to filter low-confidence actions and ensure the safe and stable execution of long-cycle and complex operations.
ACE-Ego's core advantages
- Leading performanceRoboCasa GR1 TableTop 72.8% refreshes the state-of-the-art (SOTA) and significantly surpasses mainstream models such as π0.5 (37.0%) and GR00T-N1.6 (47.6%).
- Cross-morphological generalizationThe one-brain-multiple-type architecture supports the same model to adapt to different robots, reducing the deployment cost of multiple entities.
- Complex task capabilityIt performs stably in high-contact, long-cycle retail operations, covering the entire chain from product sorting to packaging and fulfillment.
- High data efficiencyBased on pre-training with human videos, it reduces reliance on expensive robot demonstration data.
ACE-Ego project address
- Project official websitehttps://acerobotics-vla.github.io/ACE-Ego/
- GitHub repositoryhttps://github.com/ACERobotics-VLA/ACE-Ego-0
- Technical Papers: https://acerobotics-vla.github.io/ACE-Ego-0/paper.html
Comparison of ACE-Ego with similar competing products
| Dimension | ACE-Ego | π0.5 |
|---|---|---|
| Publisher | DaXiao Robotics × CUHK MMLab | Physical Intelligence |
| position | One Brain Multiple Body Operation (VLA) | General Flow Matching VLA Model |
| RoboCasa GR1 | 72.8%(SOTA) | 37.0% |
| pre-training data | 6.0K+ hours of human video + robot data | Large-scale multi-body robot data |
| Core Features | Human-centered, camera-space motion, morphological condition coding | Stream matching action generation, large-scale pre-training |
| Open source status | open source | Closed-source (API/model weights are limited) |
| Shape adaptation | Native support for multiple brain types | Individual fine-tuning is required for each entity. |
Application scenarios of ACE-Ego
- Retail fulfillmentAutomation of offline retail operations such as product sorting, plastic bag packaging, and shoe box packing in supermarkets, convenience stores, and other similar settings.
- Logistics warehousingIntelligentization of logistics processes requiring close contact, such as sorting, packing, and palletizing of goods within the warehouse.
- Home servicesAssistance with daily household chores such as organizing, tidying up, and simple cleaning in the home environment.
- Business ServicesItem handling, display maintenance, and countertop organization in shopping malls, hotels, restaurants, and other similar settings.
- Industrial manufacturingPrecision industrial operations such as component assembly, tool handling, and material transfer on the production line.