NitroGen - A general-purpose game AI model developed by NVIDIA in collaboration with Stanford University and others.

What is NitroGen?

NitroGen is an open-source, general-purpose game AI model developed by NVIDIA in collaboration with Stanford University, Caltech, and other institutions. It is trained on 40,000 hours of video data covering over 1,000 games and employs a large-scale behavior cloning method. The model takes game video frames as input and outputs controller input signals, making it suitable for various game types. In unseen games, NitroGen achieved a 52% improvement in task success rate compared to models trained from scratch.

NitroGen's main functions

• Multi-game compatibilityIt can play over 1,000 games, covering various genres such as role-playing, platforming, battle royale, and racing, including both 2D and 3D games.
• Input/OutputIt directly uses game video frames as input and outputs realistic gamepad operation signals, naturally adapting to all games that support gamepads.
• Post-training abilityWhen faced with a new game, there is no need to learn the rules from scratch. With just a few minor adjustments or light adaptations, you can quickly get started and it has the potential to be generalized across games.
• PerformanceIn procedurally generated game worlds and new games never encountered before, the task success rate achieved a relative improvement of 52% compared to a model trained from scratch.
• Open source sharingThe research findings, including pre-trained model weights, a complete action dataset, and related code, have been made available to the public to facilitate further research and development by developers.

NitroGen's technical principles

• Video frame-based input and outputNitroGen takes game video frames as input, analyzes the game screen to understand the game state, and outputs real gamepad operation signals. This end-to-end processing method allows it to be directly adapted to various games that support gamepads.
• Large-scale behavioral cloning trainingThe model was trained using over 40,000 hours of gameplay videos from game streamers, covering more than 1,000 games. By mimicking the actions of human players, the model learns the gameplay and strategies for different game types.
• Post-training and fine-tuning capabilitiesNitroGen supports post-training, so when facing a new game, there is no need to learn the rules from scratch. With just a few minor adjustments or lightweight adaptations, you can quickly get started and has the potential to generalize across games.
• Underlying architecture transformationIts underlying architecture is GROOT N1.5, which was originally designed for robotics and adapted for the gaming industry. This architecture's flexibility and scalability provide the foundation for the model's multi-game adaptability and generalization capabilities.
• Multi-task learning and generalizationNitroGen learns through multi-task learning, enabling the transfer of knowledge and skills between different games, allowing it to quickly adapt to and excel in new games.

NitroGen's project address

• Project official websitehttps://nitrogen.minedojo.org
• Github repository: https://github.com/MineDojo/NitroGen
• HuggingFace model libraryhttps://huggingface.co/nvidia/NitroGen
• Paper address: https://nitrogen.minedojo.org/assets/documents/nitrogen.pdf
• HuggingFace datasethttps://huggingface.co/datasets/nvidia/NitroGen

Applications of NitroGen

• Game Development and TestingIt helps game developers quickly test the gameplay and mechanics of new games, using AI to simulate player behavior, identify potential problems in advance, and optimize the game experience.
• Game content creationIt provides inspiration and materials for game content creators, using AI-generated gameplay and controls to assist in creating game guides, tutorials, or entertainment videos.
• Player assistance and trainingAs an auxiliary tool for players, it provides operation suggestions or demonstrations to help players improve their game skills, and is especially suitable for novice players to quickly get started with complex games.
• Game Research and AnalysisIt is used for game behavior research to analyze players' operation patterns and decision-making processes in different games, providing data support for game design and user experience research.
• Cross-game generalization researchAs a platform for researching general artificial intelligence and cross-game generalization capabilities, it promotes research on the adaptability and generalization capabilities of AI in different game environments.
• Education and TrainingIn the education field, it is used to develop educational games or simulation training environments, using AI to simulate real-world scenarios to improve learning outcomes and training efficiency.