GaussianCity - A 3D city generation framework developed by the S-Lab team at Nanyang Technological University.
GaussianCity is an efficient, borderless 3D city generation framework developed by the S-Lab team at Nanyang Technological University, based on 3D Gaussian rendering (3D-GS) technology. It introduces a compact BEV-Point representation method, minimizing the scene's video memory (VRAM)...
What is GaussianCity?
GaussianCity is an efficient, borderless 3D city generation framework developed by the S-Lab team at Nanyang Technological University, based on 3D Gaussian rendering (3D-GS) technology. It introduces a compact BEV-Point representation method, keeping the scene's VRAM usage constant and addressing the problem of excessive VRAM and storage requirements in large-scale scene generation. GaussianCity introduces a spatially aware BEV-Point Decoder, which captures the structural and contextual features of BEV points based on point serializers and point transformers, generating high-quality 3D Gaussian properties. GaussianCity performs exceptionally well in 3D city generation tasks from both drone and street view perspectives, achieving a 60x speed improvement compared to existing methods (such as CityDreamer), reaching new heights in both generation quality and efficiency.
GaussianCity's main functions
- Efficiently generate large-scale 3D citiesIt can quickly generate realistic, borderless urban environments, supporting diverse scenes from drone perspectives to street view perspectives.
- Low video memory and storage requirementsBased on the compact BEV-Point representation method, it keeps the video memory and storage requirements at a low level, solving the video memory bottleneck problem of traditional methods in large-scale scene generation.
- High-quality visual effects: Use the spatially aware BEV-Point Decoder to generate highly detailed and realistic 3D cities, supporting stylized editing and local modifications.
- Real-time rendering and interactionIt supports real-time rendering and interaction, quickly generating high-quality 3D city images, suitable for games, animation, virtual reality and other scenarios.
GaussianCity's technical principles
- 3D Gaussian rendering (3D-GS)The scene objects and structures are represented by a 3D Gaussian distribution, and GPU-accelerated rendering enables efficient 3D scene generation.
- BEV-Point meansThe point information in the 3D scene is decomposed into positional and style attributes, and compressed based on the bird's-eye view (BEV) and style lookup table to ensure that the amount of video memory used does not increase with the size of the scene.
- Spatial Awareness BEV-Point DecoderBased on point serializers and point transformers, it captures the structure and context information of BEV points, generates 3D Gaussian properties, and achieves high-quality 3D scene rendering.
- High-efficiency rendering processThe system combines a position encoder and a modulation MLP to generate 3D Gaussian attributes, which are then rendered into the final image using a Gaussian rasterizer.
GaussianCity project address
- GitHub repository:https://github.com/hzxie/GaussianCity
- HuggingFace model library:https://huggingface.co/spaces/hzxie/gaussian-city
- arXiv technical paper:https://arxiv.org/pdf/2406.06526
Application scenarios of GaussianCity
- Gaming and Virtual Reality (VR)It can quickly generate realistic virtual cities, support real-time rendering and interaction, and reduce development costs and time.
- Animation and film productionIt efficiently generates complex urban scenes, supports stylized rendering, and improves production efficiency.
- Urban planning and architectural designQuickly generate city layouts and architectural backgrounds to aid in the visualization of planning and design.
- Autonomous driving and traffic simulationGenerate realistic traffic scenarios for algorithm testing and traffic flow analysis.
- Geographic Information System (GIS)): Quickly converts geographic data into 3D visualization scenes, supporting city modeling and analysis.