RoboCOIN - An open-source embodied intelligence dataset developed by Zhiyuan in collaboration with multiple universities.

What is RoboCOIN?

RoboCOIN is an embodied intelligence dataset released by the Beijing Academy of Artificial Intelligence in collaboration with several universities and enterprises. It contains over 180,000 demonstration datasets, covering 421 tasks and 16 different scenarios, such as home, business, and factory. The data was collected from 15 different robotic platforms, including dual-arm robots, semi-humanoid robots, and humanoid robots, providing multi-view RGB and depth images as well as detailed kinematic states. RoboCOIN also constructs a hierarchical capability pyramid, with multi-resolution annotations from trajectory-level concepts to frame-level kinematics, enabling the model to perform structured learning.

RoboCOIN's main functions

• Large-scale datasetsIt provides over 180,000 demonstration data points, covering 421 tasks and 16 different scenarios, and supports multiple robot platforms, providing a rich data foundation for embodied intelligence research.
• Multimodal dataIt includes multi-view RGB and depth images, as well as detailed kinematic states, and supports the fusion of data from multiple sensors to meet different research needs.
• Hierarchical annotation system: Construct multi-resolution annotations from trajectory level to frame level, support structured learning from global planning to precise control, and improve the generalization and adaptability of the model.
• CoRobot frameworkIt provides the RTML quality assessment language, an automated annotation toolchain, and a multi-embodied management platform to facilitate efficient data annotation and model training.
• Open source and collaborationThe datasets, toolchains, and technical reports are all open source, supporting free use by individual developers, research institutions, and enterprises, and promoting collaborative innovation across the industry.

RoboCOIN's technical principles

• Multi-platform data collectionData is collected from various robotic platforms (such as dual-arm robots and humanoid robots) to cover different tasks and scenarios, ensuring the diversity and broad applicability of the data.
• Multimodal data fusionIt integrates data from multiple sensors, including RGB images, depth images, and kinematic states, to provide rich information input for the model and enhance its understanding of the environment and tasks.
• Layered annotation methodIt adopts a hierarchical annotation system from trajectory level to frame level to help the model learn task planning and motion control at different levels, realizing structured learning from macro to micro.
• Automated annotation toolsBy using an automated annotation toolchain, we can improve the efficiency and quality of data annotation, reduce the cost of manual annotation, and accelerate the construction and updating of datasets.
• Unified Management PlatformBy utilizing the multi-avatar management platform within the CoRobot framework, unified management and scheduling of different robot platforms can be achieved, supporting large-scale data collection and model training.
• Quality assessment languageThe RTML quality assessment language is introduced to standardize the evaluation of data annotation and model performance, ensuring data quality and model reliability.

RoboCOIN's project address

• Project official websitehttps://flagopen.github.io/RoboCOIN/
• Github repositoryhttps://github.com/FlagOpen/RoboCOIN
• arXiv technical paper: https://arxiv.org/pdf/2511.17441

Application scenarios of RoboCOIN

• Home service robotsTo help robots better understand and perform various tasks in the home environment, such as cleaning, tidying, and moving items, thereby improving the practicality and user experience of home service robots.
• Commercial service robotsIn commercial settings such as shopping malls, hotels, and restaurants, robots can be supported in completing tasks such as greeting guests, guiding customers, and delivering goods, thereby improving the efficiency and quality of commercial services.
• Industrial manufacturingUsed for task planning and operation execution of industrial robots in complex production environments, such as parts assembly and material handling, to enhance the flexibility and adaptability of industrial robots.
• Medical assistanceIn medical settings, assistive robots can perform tasks such as drug delivery, ward cleaning, and rehabilitation assistance, thereby improving the automation level and efficiency of medical services.
• Education and ResearchIt provides universities and research institutions with abundant data resources and experimental platforms, supports teaching and research related to embodied intelligence, and promotes the development of artificial intelligence technology.
• Logistics and WarehousingIn logistics centers and warehouses, robots are used to complete tasks such as sorting, handling, and organizing shelves, thereby improving logistics efficiency and accuracy.