AB
AB
AiBoss

All Categories

project

Genie 2 - Google DeepMind's latest generation of foundational world models

Genie 2 is a new generation of large-scale base world models from DeepMind, capable of generating interactive 3D game worlds up to one minute long from a single image. Genie 2 can simulate complex dynamics such as object interactions, character animations, and physics effects...

What is Genie 2?

Genie 2 is DeepMind's next-generation large-scale base world model, capable of generating an interactive 3D game world lasting up to one minute from a single image. Genie 2 can simulate complex dynamics such as object interactions, character animations, and physical effects, and supports operation with both keyboard and mouse. Genie 2 features long-term memory, remembering and accurately recreating scenes that have temporarily left the viewport. Genie 2 can create logically consistent new scene content in real time, maintaining the consistency of the entire world for up to one minute.

Main features of Genie 2

  • 3D world generated from imagesGenie 2 can generate interactive 3D game worlds from a single image, with a maximum play time of 1 minute.
  • motion controlThe model can respond to keyboard and mouse input, and recognize and move the character correctly.
  • Generate counterfactual scenariosBased on the same starting screen, multiple different development paths can be created.
  • Long-term memoryIt can remember scenes that are temporarily out of view and accurately reproduce them when the scene re-enters the field of view.
  • Continuously generating new scenes: Create new, logically consistent scene content in real time during the process, while maintaining world consistency.
  • Diverse Environment GenerationIt can generate a variety of different viewing perspectives, such as first-person perspective, isometric perspective, or third-person driving perspective.
  • 3D Structure CreationCreate complex 3D visual scenes.
  • Object properties and interactionsModel various object interactions, such as balloons bursting, doors opening, and shooting explosive barrels.

The technical principles of Genie 2

  • Autoregressive latent variable diffusion model: An autoregressive latent variable diffusion model trained on a large-scale video dataset.
  • Video frame processingThe latent variable frames of the video are first processed based on an autoencoder and then passed to a large-scale Transformer dynamic model.
  • Causal mask trainingThe Transformer model is trained based on causal masks, similar to the masking method used in large language models.
  • Regressive samplingDuring the inference phase, Genie 2 samples in an autoregressive manner, frame by frame using a single action and previous latent variable frames.
  • No classifier guidanceIn motion control, classifier-free guidance can be used to improve the controllability of motion.

Genie 2 project address

Application scenarios of Genie 2

  • Agent training and evaluation: Create complex virtual environments to train and test the performance and decision-making capabilities of AI agents in simulated environments.
  • Game developmentWith the dynamic world generated by Genie 2, game developers can design richer and more interactive game content, enhancing player immersion.
  • Simulation and TrainingIt can simulate battlefield environments in military training or historical events in education, providing a realistic simulation experience for training and learning.
  • Robot LearningAs a platform for robot training, it simulates different environments and situations to help robots learn how to navigate and operate in the real world.
  • Virtual Reality (VR) and Augmented Reality (AR)Combining VR and AR technologies to create virtual environments for entertainment, education, or professional training, providing an immersive experience.
Back