GWM-1 - Runway's first universal world model

What is GWM-1?

GWM-1 is Runway's first general-purpose world model, built on Gen-4.5 and employing an autoregressive architecture to predict video content frame-by-frame, enabling real-time interaction. The model includes three variants: GWM Worlds for real-time environment simulation, generating immersive, infinitely explorable spaces; GWM Avatars, an audio-driven interactive video generation model that simulates natural human movements and expressions; and GWM Robotics, a robot training simulator that generates synthetic data to accelerate robot development. GWM-1, by simulating real-world interactions, drives the shift in AI from passive generation to active simulation, contributing to the development of fields such as gaming, education, and robotics.

Main functions of GWM-1

• Real-time interaction and simulationThe GWM-1 can generate and simulate virtual worlds in real time, allowing users to interact with the virtual environment through actions such as camera movement, robot commands, and voice.
• Multi-domain applications:
  • GWM WorldsUsed for real-time environment simulation, generating immersive, infinitely explorable spaces suitable for games, virtual reality, and simulation training.
  • GWM AvatarsAudio-driven interactive avatar generation that simulates natural human expressions and movements can be used for virtual meetings, education, and entertainment.
  • GWM RoboticsAs a robot training simulator, it generates synthetic data to accelerate robot development and strategy evaluation.
• Supports synthetic data generationBy simulating different scenarios and conditions, synthetic data is generated for training and evaluating AI models, thereby improving the models' generalization ability and robustness.
• Highly customizableUsers can fine-tune the model according to their needs to adapt it to specific domains and tasks.

Technical principles of GWM-1

• Autoregressive architectureGWM-1 is an autoregressive model built on Gen-4.5. It achieves dynamic simulation by predicting video content frame by frame and generating the next frame using information from the current frame.
• Pixel-level predictionThe model learns physics, lighting, geometry, and causal relationships directly from video frames, building an understanding of the world through pixel-level predictions. This approach enables the model to generate coherent and physically consistent virtual environments.
• Multimodal input and interactionThe GWM-1 supports multiple input methods (such as text prompts, images, audio, etc.) and enables interaction with the virtual environment through action conditions (such as camera posture, robot commands, etc.).
• Large-scale data trainingThe model is trained on large-scale, high-quality data to gain a deep understanding of how the world works and demonstrates good generalization ability in different scenarios.
• Synthetic Data and Strategy EvaluationIn the field of robotics, GWM-1 helps robots rehearse their behavior in virtual environments by generating synthetic data, assessing the reliability of strategies, and accelerating development and optimization.

Project address of GWM-1

• Project official websitehttps://runwayml.com/research/introducing-runway-gwm-1

Application scenarios of GWM-1

• An infinitely explorable worldGWM Worlds can generate immersive, infinitely expandable virtual environments, eliminating the need for developers to manually design each scene, thus saving significant time and costs.
• Immersive virtual environmentGWM Worlds can generate complex virtual scenes in real time, allowing users to freely explore in VR. It is suitable for scenarios such as virtual tourism and virtual education.
• Virtual Meetings and CollaborationGWM Avatars can generate realistic virtual characters for virtual meetings and remote collaboration, improving communication efficiency and experience.
• Synthetic data generationGWM Robotics can generate synthetic data for robot training and policy evaluation, helping robots to rehearse their behavior in virtual environments and improve their performance in the real world.
• High-risk scenario simulationBy simulating high-risk or difficult-to-reproduce real-world scenarios, robots can learn and optimize their behavioral strategies in advance, reducing risks in actual testing.