UnifoLM-VLA-0 - A large-scale, open-source general-purpose robot manipulation model from Unitree Robotics.
UnifoLM-VLA-0 is a general-purpose vision-language-action (VLA) large model open-sourced by Unitree Robotics, based on the Qwen2.5-VL-7B architecture for continuous pre-training.
What is UnifoLM-VLA-0?
UnifoLM-VLA-0 is a general-purpose vision-language-action (VLA) model open-sourced by Unitree Robotics, continuously pre-trained based on the Qwen2.5-VL-7B architecture. By fusing multi-dimensional supervision signals such as 2D/3D spatial perception and trajectory prediction, the model evolves from "visual language understanding" to "embodied intelligent agent." Using a single strategy, the model can complete 12 complex humanoid robot tasks, including tidying a table, folding towels, and sorting fruit. In the LIBERO simulation benchmark, it achieves an average accuracy of 98.7%, demonstrating powerful spatial reasoning capabilities and cross-task generalization performance.
Main functions of UnifoLM-VLA-0
- General Robot OperationAs an end-to-end vision-language-action model, UnifoLM-VLA-0 can control a humanoid robot to complete complex tasks through natural language commands, including 12 types of multi-step long-range tasks such as organizing items, folding towels, sorting fruits, putting tools back in place, and wiping the table.
- Spatial perception and reasoningThe model possesses powerful 2D/3D spatial understanding capabilities, including object detection and segmentation, 3D bounding box prediction, spatial relationship reasoning (such as "the pencil on the left"), affordance reasoning (capable of grasping position determination), and trajectory planning.
- Single-strategy multitaskingIt can generalize to different scenarios and tasks with only a single model weight, without the need for separate training for each task, demonstrating excellent cross-task transfer capabilities.
Technical Principles of UnifoLM-VLA-0
- Model ArchitectureBased on the open-source Qwen2.5-VL-7B visual language model, an Action Head has been added to build an end-to-end vision-language-action architecture, enabling the direct output of robot control actions from visual perception and natural language understanding.
- Continuous pre-trainingThe model is continuously pre-trained on a multi-task dataset covering robots and general scenarios, integrating diverse supervision signals such as 2D detection and segmentation, hierarchical task decomposition, 3D object detection, spatial reasoning, and trajectory prediction to enhance the model's multimodal perception and physical understanding capabilities.
- motion modelingThe model introduces an action block prediction mechanism and applies forward and inverse dynamic constraints to uniformly model the action sequence, enabling the model to deeply understand the physical interaction dynamics between the robot and the object, and supporting long-range action planning and decision-making.
- Space EnhancementBy deeply integrating textual instructions with 2D/3D spatial details, it establishes a precise alignment between semantic logic and geometric space, significantly improving spatial perception and geometric understanding capabilities, and meeting the requirements of operational tasks for instruction comprehension and spatial reasoning.
The project address for UnifoLM-VLA-0
- Project official websitehttps://unigen-x.github.io/unifolm-vla.github.io/
- GitHub repositoryhttps://github.com/unitreerobotics/unifolm-vla
Application scenarios of UnifoLM-VLA-0
- Home servicesThe model can perform everyday household chores such as tidying up the desktop, folding towels, and wiping stains.
- Office AssistanceThe model can perform office environment maintenance tasks such as tidying up stationery, organizing school bags, and putting tools away.
- HealthcareIt supports medical assistance scenarios that require precise operations, such as opening medicine bottles and dispensing medicines.
- Education and TrainingIt can be used for structured cognitive teaching demonstrations such as sorting items by color and stacking building blocks.
- Industrial sortingThe model supports automated sorting, which categorizes and places items such as fruits and parts into designated areas according to rules.