project
Bolt3D - A 3D scene generation technology developed by Oxford University in collaboration with Google.
Bolt3D is a new 3D scene generation technology jointly developed by Google Research, the VGG team at Oxford University, and Google DeepMind. It's a potential diffusion model that can generate scenes directly from a single GPU in less than seven seconds...
What is Bolt3D?
Bolt3D is a new 3D scene generation technology jointly developed by Google Research, the VGG team at Oxford University, and Google DeepMind. It's a potential diffusion model that can directly sample a 3D scene representation from one or more images on a single GPU in less than seven seconds. On an NVIDIA H100 graphics processing unit, Bolt3D can process a photo into a complete 3D scene in just 6.25 seconds.
Bolt3D's main functions
- Quickly generate 3D scenesBolt3D is a feedforward generation method that can directly sample 3D scene representations from one or more input images. It generates images extremely quickly, taking only 6.25 seconds on a single GPU.
- Multi-perspective input and generalization capabilityIt supports different numbers of input images, from single-view to multi-view, can generate content of unobserved areas, and has good generalization ability.
- High-fidelity 3D scene representationIt uses Gaussian sputtering technology to store data and constructs 3D scenes by arranging 3D Gaussian functions in a 2D grid. Each function records position, color, transparency and spatial information, resulting in high-quality 3D scenes.
- Real-time interaction and applicationsUsers can view and render the generated 3D scenes in real time in a browser, which has broad application prospects, such as game development, virtual reality, augmented reality, architectural design, film and television production, etc.
Bolt3D's technical principles
- Geometric multi-view potential diffusion modelA multi-view latent diffusion model was trained to jointly model images and 3D point maps. The model takes one or more images and their camera poses as input and learns to capture the joint distribution of the target image, target point map, and source view point map.
- Geometry VAEA geometric VAE was trained, jointly encoding a point map and a camera ray map of a view into a geometric latent feature. The model is optimized by minimizing a combination of a standard VAE objective and a specific geometric loss, enabling high-precision compression of the point map.
- Gaussian head modelGiven a camera and the generated images and point maps, train a multi-view feedforward Gaussian head model and output a 3D Gaussian thinned color, opacity, and covariance matrix stored in the scatter image.
- Large-scale multi-view consistent datasetTo train Bolt3D, a large-scale, multi-view consistent 3D geometry and appearance dataset was created, generated by applying state-of-the-art dense reconstruction techniques to existing multi-view image datasets.
- Three-stage training processThe training process employs a three-stage approach: first, a geometric variational autoencoder (GEOAE) is trained; then, a Gaussian head model is trained; and finally, a latent diffusion model is trained.
Bolt3D project address
- Project official website:https://szymanowiczs.github.io/bolt3d
- arXiv technical paper:https://arxiv.org/pdf/2503.14445
Application scenarios of Bolt3D
- Game developmentQuickly generate 3D scenes in games, reducing development time and costs.
- Virtual Reality and Augmented RealityProvides real-time 3D scene generation for VR and AR applications, enhancing the user experience.
- Architectural DesignQuickly generate 3D models of buildings for easy design and presentation.
- Film and television productionUsed for special effects production in movies and TV series, quickly generating complex 3D scenes.