HoloPart - A joint project between HKU and VAST, providing open-source generation of complete, editable 3D models.
HoloPart is a novel diffusion model developed by the University of Hong Kong and the VAST team. It supports the decomposition of 3D objects into complete, editable semantic parts, even when the parts are occluded.
What is HoloPart?
HoloPart, a novel diffusion model developed by the University of Hong Kong and the VAST team, supports the decomposition of 3D objects into complete, editable semantic parts, even when parts are occluded. HoloPart is based on a two-stage approach, using local attention and global contextual attention mechanisms to ensure consistency between part details and overall shape. HoloPart significantly outperforms existing methods on the ABO and PartObjaverse-Tiny datasets, offering new possibilities for downstream applications such as geometry editing, material editing, and animation.
HoloPart's main functions
- Implicit segmentation of 3D partsIt can identify visible surface fragments, support the completion of occluded parts, and generate complete 3D parts.
- Geometric super-resolutionSupports super-resolution reconstruction of geometric details.
- Downstream application supportIt supports a variety of downstream applications, including geometry editing, material editing, animation production, and geometry processing.
HoloPart's technical principles
- Two-stage approach:
- Initial segmentationUse existing 3D part segmentation techniques (such as SAMPart3D) to obtain initial, incomplete part fragments (surface fragments).
- Parts replenishmentIt uses PartComp (a diffusion-based network) to complete fragments into complete 3D parts.
- diffusion modelPartComp is a diffusion-based network that captures fine-grained geometric details of parts, ensuring accurate reconstruction of local features. It uses contextual information about the overall shape to ensure that the completed part remains geometrically and semantically consistent with the overall shape.
- Data pre-training and fine-tuningWe pre-trained a general 3D shape representation on large-scale complete 3D shape data using a variational autoencoder (VAE) and a diffusion model. We then fine-tuned the pre-trained model on limited part data to adapt it to part completion tasks, overcoming the challenge of data scarcity.
HoloPart's project address
- Project official website:https://vast-ai-research.github.io/HoloPart/
- GitHub repository:https://github.com/VAST-AI-Research/HoloPart
- HuggingFace model library:https://huggingface.co/VAST-AI/HoloPart
- arXiv technical paper:https://arxiv.org/pdf/2504.07943
- Experience the demo online:https://huggingface.co/spaces/VAST-AI/HoloPart
Application scenarios of HoloPart
- Geometric EditingModify the size, shape, and position of parts to meet design requirements.
- Material allocationAdding different materials to parts enhances their visual appeal.
- Animation ProductionAllow parts to move independently, such as a wheel turning, to improve the flexibility of animation.
- Geometric processing: Optimize the mesh generation of parts to improve model quality.
- Data generationIt provides high-quality part data for 3D model training, enriching creative materials.