HOVER - NVIDIA's 1.5M miniature model of a universal humanoid robot function controller.

What is HOVER?

HOVER, a 1.5M miniature model from NVIDIA, stands for "Humanoid Versatile Controller," a multi-functional, whole-body neural universal controller for humanoid robots. The model uses 1.5 million parameters to control complex robot movements, adapting to diverse tasks such as navigation, desktop operation, and mobile operation, each requiring a unique control mode. HOVER's core innovation lies in using whole-body motion mimicry as a universal abstraction for various tasks, providing general motor skills for learning multiple whole-body control modes. Based on a multi-modal policy distillation framework, it integrates different control modes into a unified policy, achieving seamless switching between modes while retaining the advantages of each mode, improving the efficiency and flexibility of future humanoid robot applications.

HOVER's main functions

• Multi-mode controlHOVER can seamlessly switch between different control modes, such as navigation, desktop operation, and mobile operation, each with specific control requirements.
• Kinematic position trackingHOVER supports 3D position tracking of key parts of the robot, enabling precise motion replication.
• Joint angle trackingHOVER can track the target joint angle of each robot motor to achieve complex limb movements.
• Root trackingHOVER can track a robot’s root velocity, height, and orientation, which is crucial for maintaining balance and performing dynamic movements.
• Unified command spaceHOVER designs a unified command space that adapts to various control devices, such as joysticks, motion capture systems, exoskeletons, and virtual reality headsets.
• Strategy distillationHOVER extracts and distills from Oracle strategies, integrating multiple control skills into a single "universal strategy" to improve efficiency and flexibility.

HOVER's technical principles

• Proprioception and command shieldingHOVER determines task commands based on proprioception (the robot's own state perception) and command masking (selectively activating different command spaces), supporting independent control of the robot's upper and lower body.
• Multi-mode strategy distillation frameworkHOVER uses strategy distillation technology to transform Oracle strategies learned from large-scale human motion data into a single strategy that can handle multiple control modes.
• DAgger frameworkHOVER uses the DAgger (Dataset Aggregation) framework to align student policies and Oracle policies, and optimizes student policies based on supervised learning.
• Simulation trainingHOVER training is performed in the NVIDIA Isaac simulation suite, which is GPU-accelerated and can quickly simulate long-term training, transferring the model to the real world without additional fine-tuning.
• Unified command space designHOVER's command space design is both general and atomic, covering most existing control configurations and allowing for arbitrary combinations to support various modes.
• Motion redirectionHOVER can redirect human motion datasets into robot motion datasets based on matching key points and optimizing parameters.

HOVER's project address

• Project official website:hover-versatile-humanoid.github.io
• arXiv technical paper:https://arxiv.org/pdf/2410.21229

Application scenarios of HOVER

• Navigation and MobilityHOVER can control humanoid robots to navigate and move in complex environments, such as walking on uneven ground or moving through narrow spaces, making it suitable for search and rescue, tour guide services and other scenarios.
• Desktop operationIn desktop tasks requiring fine motor skills, such as assembling electronic components or performing laboratory work, HOVER can precisely control the robot's upper limbs and hand joints to achieve complex hand-eye coordination operations.
• Loco-manipulationCombining movement and manipulation tasks, such as grasping and carrying objects during movement, HOVER can coordinate the robot's full-body movements to achieve smooth handling and manipulation tasks.
• Human-computer interactionHOVER supports human-computer interaction with various input devices (such as VR headsets, motion capture systems, and exoskeletons), enabling robots to mimic human movements and behaviors, making it suitable for entertainment, education, and training.
• Remote operation and simulationIn remote operation scenarios, HOVER can respond to the operator's instructions in real time and execute precise remote tasks, such as remote surgical assistance or operations in hazardous environments.