Wall-OSS-0.5 - An open-source, domestically developed embodied intelligence model for independent variable robots.

What is Wall-OSS-0.5?

Wall-OSS-0.5 is an open-source, domestically developed embodied intelligence model for independent variables robots. It belongs to the Vision-Language-Motion (VLA) large-scale model category and possesses the "zero-shot" capability to be deployed on real robots without task fine-tuning. Through an innovative Gradient-Bridge collaborative training method, the model integrates motion capabilities into the base model backbone, enabling the robot to directly perform 17 real-world tasks such as handling and sorting, with a completion rate exceeding 80% for four of these tasks.

Main functions of Wall-OSS-0.5

• Zero-sample real robot operationPre-trained checkpoints can be directly deployed to physical hardware, performing operations such as grabbing, sorting, and organizing without needing to fine-tune them for specific tasks, thus achieving "pre-training for deployment".
• Unified adaptation across multiple formsA single pre-trained checkpoint can adapt to more than 20 robot forms, such as desktop dual arms and mobile operations, breaking the limitation that traditional models need to be trained separately for each type of hardware.
• Three-objective coordinated trainingBy complementing and optimizing discrete action prediction, multimodal prediction, and continuous flow matching, a "gradient bridge" is formed, allowing action learning and visual language understanding to improve synergistically without interfering with each other.
• Hybrid Expert Architecture (MoT)Visual-language tokens are routed and processed by VL Experts, while continuous action computation is handled by Action Experts. A joint attention mechanism enables end-to-end gradient flow, taking into account both language understanding and action generation.
• Visual language ability retentionMotion training does not erode basic VLM abilities, improves entity grounding ability by 21.8 percentage points, and fully preserves general visual-language comprehension and reasoning abilities.
• High-efficiency fine-tuning adaptationAfter fine-tuning on the LeRobot dataset, the average progress of 15 real robot tasks reached 60.5%, outperforming similar models π0.5 by 17.5 percentage points.
• Strong generalization and long-term executionIt maintains high progress on 3 unseen tasks (such as manipulating deformable objects) and supports multi-stage continuous operations (such as long-term tasks like tightening ropes, folding rags, and organizing fruit baskets).

The technical principles of Wall-OSS-0.5

• 4B Parameter VLA ArchitectureBased on the Qwen2.5-VL visual language model with 3 billion parameters as the backbone, and superimposed with action generation components, the total number of parameters reaches 4 billion, realizing end-to-end unification of visual perception, language understanding and robot action generation.
• Hybrid Expert Routing (MoT)The architecture adopts Mixture-of-Experts, where visual-language tokens are routed through VL Experts, and continuous action computation is handled by independent Action Experts. The two are connected through a joint attention mechanism to achieve end-to-end gradient flow, avoiding interference between action training and language understanding.
• Gradient bridging collaborative trainingInnovatively, discrete action prediction, multimodal prediction, and continuous flow matching are used as complementary optimization objectives for collaborative training, forming a "gradient bridge" that enables action policy learning and visual language pre-training knowledge to reinforce each other rather than conflict.
• Continuous Stream Matching Action GenerationUsing Flow Matching technology to model the continuous motion space, compared with traditional discretization or regression methods, it can generate smoother, higher-dimensional robot control signals more accurately.
• Pre-training as Deployment DesignBy pre-training with large-scale heterogeneous data, the model learns general physical operation priors during the pre-training stage. Checkpoints can be directly performed on real robot hardware with zero-shot inference, without the need for task-specific fine-tuning post-processing.
• Large-scale heterogeneous training dataIt covers more than 20 robot types, with each training round containing more than 1 million real robot trajectories; at the same time, it integrates 90 million multimodal corpora (including 12 million entity bridging samples), achieving a task-balanced sampling of about 60% self-collected data and 40% open source data.
• Joint attention and end-to-end optimizationVL Expert and Action Expert share an attention computation graph, ensuring that language commands, visual scenes and action outputs are deeply coupled at the feature level, and gradients can propagate bidirectionally across modalities.
• Capability decoupling and preservation mechanismThrough a three-objective collaborative and expert-separated design, motion training does not erode basic VLM capabilities, improves entity grounding ability by 21.8 percentage points, and fully preserves general visual-language understanding and reasoning capabilities.

How to use Wall-OSS-0.5

• Environmental preparationCreate a conda environment for Python 3.10, install basic dependencies such as PyTorch, and ensure that Flash Attention version 2.7.4 or above is installed to accelerate inference.
• Install LeRobot libraryClone the Hugging Face LeRobot repository, checkout to the specified compatible version, and then execute. pip install -e . Installation complete.
• Installing the Wall-X toolchain:clone wall-x Open source repository, running git submodule update --init --recursive After pulling the submodule, execute the installation command to complete the toolchain deployment.
• Download pre-trained weightsFrom Hugging Face (x-square-robot/wall-oss-0.5Download the pre-trained model checkpoints released by the official website.
• Configure robot parametersSet the degrees of freedom (DOF) configuration, model path, data path, and training hyperparameters according to the target robot type (such as desktop dual-arm, mobile operation, etc.).
• Perform fine-tuning (optional)For optimizations targeting specific tasks, run fine-tuning scripts on LeRobot format datasets (e.g., ...). bash ./workspace/lerobot_example/run.shTo improve task performance.
• Deploy to real hardwareLoad pre-trained or fine-tuned checkpoints, perform zero-shot inference or fine-tuned inference on real robot hardware, and directly output an executable robot control strategy.

The core advantages of Wall-OSS-0.5

• Deploy and use immediatelyUnlike traditional VLA models that require fine-tuning before use, pre-trained checkpoints directly generate executable robot policies.
• Highly adaptableAfter fine-tuning on 15 real robotic tasks, the average progress was 60.5%, which is 17.5 percentage points higher than π0.5's 43.0%.
• Strong generalization abilityMaintaining high progress on 3 unseen tasks (such as manipulating deformable objects), with the rope tightening task at 82%.
• Visual language ability retentionMotion training does not erode basic VLM capabilities, improves physical grounding capabilities by 21.8 percentage points, and fully preserves general VL capabilities.
• Leading in data scaleIt covers 20+ robot types, with 1 million+ trajectories per round and 90 million multimodal samples.

The project address for Wall-OSS-0.5

• Project addresshttps://x2robot.com/oss#resources
• Github repositoryhttps://github.com/X-Square-Robot/wall-x
• Paper address: https://x2robot.com/api/files/file/wall_oss_05.pdf

Wall-OSS-0.5 Comparison with Similar Competitors

Application scenarios of Wall-OSS-0.5

• Home services: Sorting building blocks, classifying fruits, organizing drawers, stacking rings, and other everyday household chores.
• Flexible object processingTasks involving manipulating deformable objects, such as tightening ropes, folding rags, and arranging flowers.
• Industrial assemblyRepetitive manufacturing processes such as precision parts picking, color sorting, and tool return.
• Arm collaborationComplex assembly, sorting, and long-range combination actions that require the coordination of both hands.
• Long-range task executionMulti-stage continuous operation, such as basket sorting and other multi-step tasks that require sustained attention.