Qwen-VLA - A universal vision-language-action model launched by Alitongyi

What is Qwen-VLA?

Qwen-VLA is a general-purpose vision-language-action model developed by Tongyi Labs. It uses Qwen3.5-4B as its visual-language backbone and is paired with a DiT action decoder with 1.15B parameters. The model unifies the three major tasks of manipulation, navigation, and trajectory prediction into a single model through a unified action trajectory prediction framework. With the help of ontology-based conditional cues, it can be adapted to 11 robot platforms simply by modifying the text description. The model outperforms dedicated models in multiple benchmarks, achieving a 26.6% success rate in zero-shot dynamic manipulation, thus propelling embodied intelligence from skill experts to general actors.

Main functions of Qwen-VLA

• Unified control across tasksA single model can handle three major tasks simultaneously: operation, navigation, and trajectory prediction, breaking the fragmented nature of traditional dedicated models.
• Cross-body plug and playSupports 11 robot platforms including WidowX, Franka Panda, Mobile ALOHA, and Galaxea R1, covering configurations such as single-arm, dual-arm, and mobile base. Switching between these platforms is as simple as modifying the text prompts.
• Zero-shot open-world generalizationIt maintains a high success rate across five unseen dimensions of distribution: color, instance, position, background, and instruction, without requiring retraining for new environments.
• Dynamic object manipulationNo dynamic training data is required; zero-shot operation on moving objects; DOMINO benchmark success rate of 26.6%, surpassing dedicated fine-tuning models.
• Long-term navigationThe task-adaptive token allocation mechanism preserves a richer episode history for long instructions, and the VLN-CE benchmark R2R of 57.5% and RxR of 59.6% surpasses dedicated navigation models.

Qwen-VLA Technical Principles

• Unified motion trajectory prediction frameworkTraditional embodied intelligence separates manipulation, navigation, and trajectory prediction into independent models, resulting in limited cross-task transferability. Qwen-VLA finds that the computational structures of these three are isomorphic—all involve "observing the scene + understanding instructions → predicting future action sequences," thus unifying their modeling. The model combines the Qwen 3.5-4B visual-language backbone with a 1.15B DiT action decoder, with data from different tasks undergoing joint supervision during the same training, enabling cross-task transferability of visual localization and spatial reasoning capabilities.
• Ontology-based cue conditionalizationTo address the differences in robot hardware, traditional solutions customize independent branches for each robot body. Qwen-VLA encodes these hardware differences into structured text prompts as a single platform interface. These prompts contain information such as robot model, number of arms, joint configuration, and control frequency, which are then processed by the VLM. The hidden states of the backbone network are concatenated with noisy action blocks and sent to the DiT decoder. During inference, only the prompts need to be replaced to switch platforms.
• Text-to-Action DiT Pre-training (T2A)The training faces the challenge of pre-training the VLM while initializing the DiT from scratch. The solution is to freeze the VLM, train the DiT separately, and input only text without providing images. This allows the DiT to first learn action distribution, text-action alignment, and ontology conditionalization. The computational cost at this stage is only 1/10 of that of multimodal training. Ablation experiments validate that 20% synthetic + 80% real data is optimal (71.1%), and no images are actually better than with images, peaking at 2,000 steps.

How to use Qwen-VLA

• Environmental preparationClone the official repository from GitHub and install the dependencies, then download the pre-trained weights.
• Model InferenceBased on the robot's hardware configuration, generate body prompts, input images, instructions, and prompts into the model, obtain action sequences, and send them to the robot for execution. Iterate in a loop to achieve closed-loop control.
• Model Training (Advanced)Training follows a four-stage process: T2A → CPT → SFT → RL, gradually improving motion generation, visual perception, task adaptation, and closed-loop optimization capabilities.
• Cross-platform deploymentWhen switching robots, you only need to modify the hardware description field in the prompt. When adapting to the new robot, you can make minor adjustments to the collected data during the SFT stage.

Qwen-VLA's core advantages

• versatility surpasses specializationThe single general-purpose model outperforms the best dedicated model in 3 out of 5 simulation benchmarks.
• Lightweight cross-body adaptationOnly text prompts need to be modified; there is no need to retrain the model for each robot.
• Efficient pre-training strategiesThe computational cost of the T2A stage is only 1/10 of that of multimodal training, thus avoiding interference with the VLM pre-training results.
• Strong OOD generalization abilityThe real-world average OOD success rate is 76.9%, surpassing π₀.₅ +35.4pp and the untrained variant +40.7pp.
• Breakthrough in zero-shot dynamic scenariosDOMINO benchmark zero-sample performance is 26.6%, outperforming the dedicated fine-tuning model PUMA by 17.2%.

Qwen-VLA's project address

• Project official website: https://qwen.ai/blog?id=qwenvla
• GitHub repositoryhttps://github.com/QwenLM/Qwen-VLA
• arXiv technical paper: https://arxiv.org/pdf/2605.30280

Comparison of Qwen-VLA with similar competing products

Application Scenarios of Qwen-VLA

• Multi-task industrial robotsThe same model can complete heterogeneous tasks such as picking, moving, and navigating to different workstations on the assembly line without having to deploy a separate model for each task.
• Service robots deployed across scenariosHome robots can perform operations in the kitchen, navigate autonomously in the living room, and plan routes in the hallway, seamlessly switching between task types.
• Scientific research and education platformResearchers can quickly validate algorithms on new hardware platforms without retraining models for each robot, simply by modifying text prompts.
• Dynamic environment operationIn logistics and warehousing, it can grab moving packages on conveyor belts without needing to re-collect training data for dynamic scenarios.
• Long-term complex instruction follow-upThe museum guide robot can perform long, multi-step tasks such as "go to Hall A to take a photo first, then bypass the crowd to Hall B".