EgoSuite-Open100K - A full-modal human behavior dataset open-sourced by Lightwheel Intelligence.

What is EgoSuite-Open100K?

EgoSuite-Open100K, launched by Lightwheel Intelligence, is the world's first open-source dataset of 100,000 hours of full-modal human behavior, targeting physical AI and embodied intelligence. The dataset contains over 15,000 first-person videos of real-world scenes, covering seven major environments including home and industry. It was captured using both head and wrist perspectives and features three layers of fine annotation: 21-joint hand pose, body posture, and event semantics. The dataset is freely available on Hugging Face and AtomGit, supporting academic and commercial use, and addresses the bottleneck of high-quality human behavior data being the most scarce resource in robot training.

Main functions of EgoSuite-Open100K

• Large-scale human behavior data collectionIt provides 100,000 hours of first-person, real human operation videos, providing ample behavioral prior data for physical AI training.
• Dual-view synchronous recordingThe system combines a head-mounted main view with a wrist close-up view to ensure the overall continuity of the movements while capturing the fine details of finger operations.
• Three-layer full-modal annotationThe model is labeled frame by frame with the 21 joint poses of the hand, the overall posture, and event-level semantic information, enabling the model to learn both "how to move" and "why to move".
• Cross-scenario generalization coverageIt covers 7 major environmental categories and 128 scene types, including home, industrial, and medical, breaking through the limitation of existing datasets that only cover the kitchen scene.
• Open source and business friendlyHugging Face and AtomGit are completely open and free to use for both academic research and business training, breaking the data monopoly.
• Unified data standardsFollowing the EgoVerse International Data Council standards, it addresses the industry pain point of data format incompatibility between different devices and teams.
• Supporting cross-body migrationThe amount of data reaches 100,000 hours, enabling the model to generalize human actions to different robot bodies.
• Closed-loop ecological foundationIt works in conjunction with RoboFinals simulation evaluation and RoboStack real deployment to form a continuous learning infrastructure of "collection-training-evaluation-feedback".

The technical principles of EgoSuite-Open100K

• Data Scale and Scaling LawBased on the physical AI Scaling Law, the dataset has reached 100,000 hours of human first-person perspective video, and is at a critical inflection point for the emergence of cross-embodied transfer capabilities.
• Dual-view acquisition architectureThe system employs a dual-view simultaneous acquisition approach, combining a head-mounted main view with a wrist close-up, to capture both the global motion context and the fine details of finger movements.
• Full-modal alignment annotation The video is fully aligned and annotated with all modalities, including the 21-joint pose of the hand, body posture, event-level semantics, and depth information, to construct a multimodal training signal.
• Standardized data formatIt follows the EgoVerse international data standard to unify the data collection format and annotation specifications, enabling data from different sources to be reused across devices and models.
• Continuous learning loopIt works in conjunction with the SimFoundry simulation platform and RoboStack deployment system to form a continuous learning loop of human data pre-training, simulation evaluation and verification, and real-world scenario feedback iteration.

Follow us on WeChat and reply with "open source",join inAI open source project discussion group

How to use EgoSuite-Open100K

• Get dataAccess Hugging Face Hub or AtomGit to download the first batch of open-source data, which can be used directly for academic or commercial model training without application.
• Select a subset of dataChoose between EgoStandard (head-mounted main view) or EgoPro (wrist close-up view) depending on the task requirements, which are suitable for full-body motion learning or fine-grasping operations, respectively.
• Analyzing multi-level annotationsUsing the accompanying 21-joint hand pose, body posture, and event-level semantic annotations, multimodal input signals are constructed for training policy models or VLA models.
• Access Training ProcessThe dataset is used as a pre-trained corpus to input into the robot policy network, and 100,000 hours of prior human behavior is used to improve the model's generalization ability in real-world scenarios.
• Combined with simulation verificationThe model trained on this dataset can be validated at low cost and reproducibly on a large scale using the Lightwheel Intelligent SimFoundry simulation platform or the RoboFinals evaluation system.