AB
AB
AiBoss

All Categories

project

CityDreamer4D - A borderless 4D city modeling framework developed by Nanyang Technological University.

CityDreamer4D is a composite generative model developed by the S-Lab team at Nanyang Technological University for generating borderless 4D cities. It separates dynamic objects (such as vehicles) from static scenes (such as buildings and roads) through three modules—...

What is CityDreamer4D?

CityDreamer4D is a combinatorial generative model developed by the S-Lab team at Nanyang Technological University for generating borderless 4D cities. It separates dynamic objects (such as vehicles) from static scenes (such as buildings and roads), generating city scenes based on an efficient bird's-eye view scene representation through three modules—a building instance generator, a vehicle instance generator, and a city background generator. The model has been built on multiple datasets, including OSM, Google Earth, and CityTopia, covering multiple viewpoints and lighting conditions. CityDreamer4D can generate realistic 4D cities and supports applications such as city stylization, local editing, and city simulation.

Main functions of CityDreamer4D

  • Borderless 4D City GenerationBy separating dynamic objects (such as vehicles) from static scenes (such as buildings and roads), CityDreamer4D can generate dynamic city scenes that incorporate both temporal and spatial dimensions. It supports generating infinitely expandable city layouts while maintaining consistency across multiple viewpoints.
  • Example editing and partial modificationCityDreamer4D supports partial editing of building and vehicle instances, such as modifying the position and style of vehicles, or adjusting the height and appearance of buildings, without affecting other scene elements.
  • Urban stylizationThe model supports stylizing the generated cities, such as converting the city style to Minecraft or cyberpunk style, to maintain consistency across multiple perspectives.
  • City SimulationCityDreamer4D can be used as a powerful city simulation tool to generate realistic 4D city scenes, supporting street view and aerial perspectives, and is suitable for applications such as autonomous driving, urban planning and virtual reality.
  • Dataset supportThe development team built multiple datasets, including OSM (OpenStreetMap), GoogleEarth, and CityTopia, covering multiple viewpoints and lighting conditions.

The technical principles of CityDreamer4D

  • Separate dynamic and static elementsCityDreamer4D separates dynamic objects (such as vehicles) from static scenes (such as buildings and roads), which are then processed by different generators.
  • Modular generationThe model consists of three main modules:
    • Building Instance Generator(Building Instance Generator): Responsible for generating building instances, using the object's center coordinate space to capture the building's compact shape.
    • Vehicle instance generator(Vehicle Instance Generator): Generates vehicle instances using object-specific coordinate space combined with periodic position encoding.
    • City background generator(City Background Generator): Generates background elements (such as roads, vegetation, etc.) and uses a generative hash grid to handle irregular textures in the background.
  • Bird's-eye view scene representationTo improve efficiency and scalability, CityDreamer4D employs an efficient Bird's-Eye View (BEV) scene representation. It uses a compact BEV map to depict city layouts, combined with a height map to represent building heights, and supports boundless expansion.
  • neural field combinationThe model generates background elements and instances (buildings and vehicles) based on background-oriented and instance-oriented neural fields. It can effectively capture the diversity and uniqueness of different elements in the city.

CityDreamer4D project address

Application Scenarios of CityDreamer4D

  • City Simulation and PlanningCityDreamer4D can be used as a powerful city simulation tool to generate realistic 4D city scenes, supporting the generation of dynamic objects (such as vehicles) and detailed environments.
  • Local editing and customizationWith its modular architecture, CityDreamer4D supports partial editing of building and vehicle instances. You can independently modify the position and style of vehicles, or adjust the height and appearance of buildings, without affecting other scene elements.
  • Urban stylizationCityDreamer4D supports city stylization, allowing you to seamlessly apply different styles, such as Minecraft or Cyberpunk, to the entire city.
  • Multi-perspective consistencyIn the generated 4D city, CityDreamer4D ensures consistency across multiple perspectives and supports the generation of street view and aerial perspectives.
Back