Being-M0 - A general motion generation model for humanoid robots jointly developed by Peking University and Renmin University.
Being-M0 is the first large-scale general motion generation model for humanoid robots, jointly developed by Peking University, Renmin University of China, and other institutions. Being-M0 is based on MotionLib, the industry's first dataset of millions of motion samples, and uses innovative Motion...
What is Being-M0?
Being-M0 is the first large-scale general motion generation model for humanoid robots, jointly launched by Peking University, Renmin University of China, and other institutions. Based on MotionLib, the industry's first dataset of millions of motion sequences, Being-M0 uses innovative MotionBook encoding technology to efficiently represent and generate motion sequences as two-dimensional images. Being-M0 validates the technical feasibility of using large data sets and large models in motion generation, significantly improving the diversity and semantic alignment accuracy of motion generation, and enabling efficient transfer from human motion to multiple humanoid robots, laying the foundation for general motion intelligence.
Main functions of Being-M0
- Text-driven action generationGenerates corresponding human action sequences based on the input natural language description.
- Action transferThe model can transfer the generated human actions to various humanoid robot platforms, such as Unitree H1, H1-2, and G1, allowing the robots to perform similar human actions.
- Multimodal data supportBeing-M0 supports multiple modalities of data, including RGB video and depth information, and can handle multi-person interactive scenarios, providing support for motion generation in complex scenarios.
The technical principle of Being-M0
- Large-scale datasets MotionLibBased on MotionLib, the industry's first dataset for generating millions of actions, the dataset contains over 1.2 million action sequences, 15 times larger than the largest existing public dataset. The dataset was collected from public datasets and online platforms using an automated processing workflow, and then annotated with fine-grained precision.
- MotionBook Encoding TechnologyBased on MotionBook encoding technology, action sequences are modeled as single-channel two-dimensional motion images, constructing independent encoding spaces along the time and joint axes to fully preserve the multidimensional structural features of motion. Dimensional reduction projection eliminates the traditional codebook lookup step, increasing the action vocabulary capacity by two orders of magnitude.
- A two-stage solution combining optimization and learningIn terms of motion transfer, a two-stage solution of optimization and learning is used. The optimization stage generates motion sequences that satisfy robot kinematic constraints based on multi-objective optimization methods; the learning stage uses a lightweight MLP network to learn the mapping relationship from human actions to humanoid robot actions, achieving efficient motion transfer.
The project address for Being-M0
- Project official website:https://beingbeyond.github.io/Being-M0/
- arXiv technical paper:https://arxiv.org/pdf/2410.03311
Application scenarios of Being-M0
- Humanoid robot controlTo enable robots to perform actions based on text instructions.
- Animation ProductionQuickly generate high-quality motion sequences to improve production efficiency.
- Virtual Reality and Augmented RealityReal-time generation of interactive actions enhances immersion.
- Human-computer interactionTo enable robots to better understand human instructions and improve collaboration efficiency.
- Movement Analysis and Rehabilitation: To assist athletes in training and patients in rehabilitation.