Thinker - UBTECH's open-source embodied intelligent visual language model

What is a Thinker?

Thinker is an open-source embodied intelligent visual language model developed by UBTECH, specifically designed for robotic applications. Its 4B parameters achieved world-leading rankings in nine authoritative benchmark tests. The model's core capabilities encompass task planning, spatial understanding, temporal reasoning, and visual localization, effectively addressing the challenge of robots "thinking but not being able to grasp" the information. The model was trained on 10 million high-quality data points refined from 2 billion original datasets, employing an automated annotation system to keep human intervention below 1%. The model has already enabled Walker S2 to achieve 99.99% operational accuracy in industrial scenarios, promoting the widespread adoption of embodied intelligence technology.

Thinker's main functions

• Task planningThinker can understand complex human commands, combine historical state memories, predict future state changes of the robot, and decompose long-term tasks into a sequence of executable subtasks.
• Spatial understandingThinker has established an egocentric coordinate system that uses the camera as the origin to define all spatial relationships, enabling the robot to accurately perceive the position and orientation of objects in three-dimensional space.
• Time UnderstandingThinker can extract key information from video history, combine past events with current instructions, accurately assess the current state, and make reasonable timing decisions.
• Visual positioningThinker can describe the position of an object using bounding boxes and precise point coordinates, providing accurate spatial guidance for the robot's grasping operations and interactions.

Thinker's technical principles

• Data buildingThinker has built a complete pipeline from raw data to high-quality training data. Faced with 2 billion noisy and poorly aligned raw data points, it uses customized rules for broad filtering and large-scale models for multi-dimensional quality scoring, refining 10 million high-quality data points. At the same time, it adopts an automated annotation system of "large-scale model-assisted annotation plus multi-model cross-validation," keeping the manual participation rate below 1%, reducing annotation costs by 99% and improving efficiency by more than 100 times.
• Model architecture designThinker employs a classic visual-language model architecture, comprising four core modules: a text segmenter, a visual encoder, a multilayer perceptron alignment layer, and a language model backbone. It achieves unified representation of vision, language, and time, enabling the model to accurately capture visual details, understand task instructions, and perform cross-modal reasoning.
• Training strategyThinker employs a two-stage training approach. The first stage involves fine-tuning on general datasets, spatial understanding datasets, and large-scale planning datasets to establish basic perception and reasoning capabilities, while also incorporating the last frame of a video as auxiliary input to enhance video understanding. The second stage involves supervised fine-tuning on industrial task datasets to adapt the model to sequence dependencies, diverse object layouts, and feedback corrections, ultimately generating planning schemes that can be executed in real-world industrial scenarios.
• Key InnovationThinker addresses the pain points of perspective confusion and missing video information in robots by proposing a simple and effective method to jointly input keyframes and complete videos during video understanding training, significantly enhancing the model's temporal understanding capabilities. Furthermore, through high-quality data filtering and task-oriented sampling, it achieves performance exceeding that of models with over 10 bytes of parameters while maintaining only 4 bytes of parameters.

Thinker's project address

• GitHub repositoryhttps://github.com/UBTECH-Robot/Thinker
• HuggingFace model libraryhttps://huggingface.co/UBTECH-Robotics/Thinker-4B
• arXiv technical paper: https://arxiv.org/pdf/2601.21199

Thinker application scenarios

• Industrial intelligent manufacturingThinker can drive humanoid robots to complete tasks such as box handling and workpiece sorting on factory production lines. The Walker S2 has achieved a 99.99% accuracy rate, effectively solving the problem of insufficient flexibility of traditional automated equipment.
• Warehousing and logistics operationsThinker enables robots to perform cargo identification, path planning, and precise grasping in dynamic warehouse environments, adapting to the diverse and frequently changing logistics needs of SKUs.
• Commercial service scenariosThinker empowers robots to provide guidance, explanation, and interactive services in public places such as shopping malls and exhibition halls, achieving natural human-computer interaction through visual language understanding.
• Complex operation tasksThinker enables robots to perform operations that require long-range planning and fine spatial awareness, such as equipment inspection, parts assembly, and multi-step experimental processes.
• Swarm intelligence collaborationThinker serves as a cognitive foundation supporting UBTECH's swarm brain network and collaborative intelligent agent Co-Agent, enabling task allocation, collaborative decision-making, and autonomous evolution among multiple robots.