WonderPlay - A dynamic 3D scene generation framework jointly developed by Stanford University and the University of Utah

What is WonderPlay?

WonderPlay is a new framework developed by Stanford University in collaboration with the University of Utah. It supports the generation of dynamic 3D scenes from single images and user-defined actions. Based on a combination of physics simulation and video generation technologies, it uses a physics solver to simulate rough 3D motion, then drives a video generator to synthesize more realistic videos, updating the dynamic 3D scene with the videos, thus achieving a closed loop between simulation and generation. WonderPlay supports various physical materials (such as rigid bodies, cloth, liquids, and gases) and various actions (such as gravity, wind, and point forces). Users can generate a rich variety of dynamic effects through simple operations and interaction with the scene.

WonderPlay's main functions

• Single-image dynamic scene generationUse an image and user-defined action input to generate a dynamic 3D scene that shows the physical consequences of the action.
• Multiple material supportIt covers a variety of physical materials, including rigid bodies, fabrics, liquids, gases, elastomers, and particles, to meet diverse needs in various scenarios.
• Action responseIt supports motion inputs such as gravity, wind, and point force, allowing users to intuitively operate and interact with the scene to generate different dynamic effects.
• Visual and physical realismIt combines the precision of physical simulation with the richness of video generation to produce dynamic scenes that conform to physical laws and are visually realistic.
• Interactive experienceEquipped with an interactive viewer, users can freely explore the generated dynamic 3D scenes, enhancing immersion.

The technical principles of WonderPlay

• Hybrid Generative SimulatorIt integrates a physics solver and a video generator, using the physics solver to simulate rough 3D dynamics, driving the video generator to synthesize realistic videos, and using the videos to update the dynamic 3D scene, thus achieving a closed loop between simulation and generation.
• Spatial variation dual-mode controlDuring the video generation stage, the video generator is controlled by dual-modal signals of motion (flow field) and appearance (RGB). The generator's responsibility is dynamically adjusted according to the scene area to ensure that the generated video is closer to the physical simulation results in terms of dynamics and appearance.
• 3D scene reconstructionThe background and objects are reconstructed from the input image. The background is represented by Fast Layered Gaussian Surfaces (FLAGS), and the objects are constructed as "topological Gaussian surfaces" with topological connectivity. The material properties of the objects are estimated to provide a basis for subsequent simulation and generation.

WonderPlay's project address

• Project official website:https://kyleleey.github.io/WonderPlay/
• arXiv technical paper:https://arxiv.org/pdf/2505.18151

Application scenarios of WonderPlay

• AR/VR scene constructionUsed to create immersive virtual environments that support dynamic interaction between users and the scene.
• Film and television special effects productionQuickly generate dynamic scene prototypes to assist in special effects production and enhance visual effects.
• Education and vocational trainingSimulate physical phenomena and working environments to enhance the practicality of teaching and training.
• Game developmentGenerate dynamic scenes and interactive effects to enhance the realism and fun of the game.
• Advertising and MarketingCreate dynamic advertising content, provide interactive experiences, and enhance audience engagement.