AB
AB
AiBoss

All Categories

project

GameGen-O - Tencent's game video generation model that automatically generates characters, scenes, actions, and events.

GameGen-O is a game video generation model launched by Tencent, based on the Transformer architecture, specifically designed for generating open-world video games. The model can simulate various functions of a game engine, including generating game characters, animations, etc.

What is GameGen-O?

GameGen-O is a game video generation model launched by Tencent, based on the Transformer architecture, specifically designed for generating open-world video games. The model can simulate various functions of a game engine, including generating game characters, dynamic environments, and complex actions. It supports interactive control, allowing users to control game content through text, action signals, and video prompts. The development of GameGen-O involved large-scale data collection and processing, creating the first open-world video game dataset (OGameData), and underwent a two-stage training process, including basic model pre-training and instruction tuning. The model's release will have a significant impact on the game development industry, reducing development costs and time while providing players with greater creative freedom.

GameGen-O's main functions

  • Character generationIt can generate various characters based on the user's text commands, such as cowboys, astronauts, and magicians.
  • Environment generationIt can create dynamic game environments that adapt to different game styles and scenarios.
  • Action generationIt supports generating complex character actions, such as driving, flying, and shooting.
  • Event generationIt can generate various events in the game, such as weather changes and natural disasters.
  • Interactive controlUsers can control the game content through text, operation signals, and video prompts, achieving an interactive gaming experience.

GameGen-O's technical principles

  • Open domain generationGameGen-O can generate various types of game elements, such as characters, environments, actions, and events, expanding the possibilities of games.
  • Interactive controllabilityThe model can generate game content and supports interactive control by users through InstructNet branches, such as changing character behavior, environment layout, and event occurrences.
  • OGameData datasetTo train GameGen-O, the research team built a large-scale open-world video game dataset, OGameData, which contains more than 4,000 hours of video clips from more than 150 games, covering a variety of game genres and styles.
  • Two-stage trainingThe model employs a two-stage training strategy. The first stage is basic model pre-training, which learns to generate high-quality game videos; the second stage is fine-tuning through instructions, giving the model the ability to generate and control content based on user commands.
  • Technological innovationGameGen-O employs a variety of advanced technologies, such as 2+1D VAE video compression, hybrid training strategies, and masked attention mechanisms, to ensure model stability and generation quality.
  • Dataset construction and training processWe collected 32,000 raw videos from the internet, which were then screened by human experts and labeled with GPT-4o to form high-quality training data. The basic training phase of the model used a variational autoencoder to compress video segments and employed a hybrid training strategy with different frame rates and resolutions.
  • InstructNetDuring the model fine-tuning phase, a trainable InstructNet was used to accept multimodal inputs, including text, action signals, and video cues, enabling interactive control of the generated content.

GameGen-O's project address

Application Scenarios of GameGen-O

  • Game Prototype CreationDevelopers can use GameGen-O to quickly create game prototypes, test different game elements, and save time and effort in building a game from scratch.
  • Environment and Scene GenerationGameGen-O can create dynamic game environments and complex scenes, adding rich visual effects to the game world.
  • Action and event generationThe model supports generating complex character movements and various events in the game, such as tsunamis, tornadoes, and fires, increasing the game's interactivity and challenge.
  • Assisting in game developmentGameGen-O can assist in the game development process by replacing some development work with AI models, thereby improving development efficiency.
  • Research and EducationFor researchers and educators, GameGen-O can serve as a research tool to help explore areas such as video game AI development, interactive control, and immersive virtual environments.
Back