PartCrafter - AI 3D model generation, supporting multi-part co-generation.

What is PartCrafter?

PartCrafter is an advanced 3D generative model capable of generating multiple semantically well-defined and geometrically diverse 3D meshes from a single RGB image. By combining latent space representations of each 3D part, a hierarchical attention mechanism is used to transfer information within and between parts, ensuring global consistency in the generated 3D models. The model is based on a pre-trained 3D Mesh Diffusion Transformer (DiT), inheriting its weights, encoder, and decoder to further enhance its generative capabilities. It enables end-to-end part-aware generation, supporting the generation of scenes ranging from single objects to complex multi-object scenarios. Experiments show that the model outperforms existing methods in generating decomposable 3D meshes, accurately generating parts that are not directly observable in the input image.

Main functions of PartCrafter

• Multi-component joint generationPartCrafter can generate multiple parts and objects from a single image at once, each with a clear semantic meaning and a unique geometric shape.
• Component-aware generationThe model represents each 3D part by combining the latent space and uses a hierarchical attention mechanism to transfer information within and between parts.
• End-to-end generationPartCrafter supports end-to-end generation from a single image to complex multi-object scenes. Users only need to provide an image, and the model can generate a complete 3D scene, including multiple individual parts and objects.
• Component-level editingThe generated 3D models support component-level editing and modification. Users can perform individual operations on the generated components, such as adjusting their position, rotating them, and scaling them, enabling more flexible 3D design and creation.
• Complex scene modelingPartCrafter is suitable for modeling complex scenes, such as mechanical structures and architectural models. It can generate multiple parts and ensure the interrelationship between these parts, resulting in more realistic and complex 3D scenes.
• Pre-trained model supportThe model is based on a pre-trained 3D mesh diffusion transformer (DiT), inheriting its weights, encoder, and decoder. This pre-training mechanism further enhances the model's generative capabilities and efficiency.

PartCrafter's technical principles

• Combined potential spaceEach 3D part is represented by a set of decoupled potential tokens, allowing the parts to evolve independently during generation while preserving part-level details.
• Hierarchical attention mechanismIt supports the flow of structured information within a single component and between all components, ensuring global consistency during the generation process.
• Pre-trained 3D Mesh Diffusion Transformer (DiT)PartCrafter inherits the pre-trained weights, encoder, and decoder, further enhancing the model's generation capabilities.
• End-to-end component sensing generationUnder the condition of a single image, PartCrafter can simultaneously denoise multiple 3D parts, realizing end-to-end part perception generation from individual objects to complex multi-object scenes.

PartCrafter's project address

• Project official website:https://wgsxm.github.io/projects/partcrafter/
• Github repository:https://github.com/wgsxm/PartCrafter
• arXiv technical paper:https://arxiv.org/pdf/2506.05573

Application scenarios of PartCrafter

• Game developmentPartCrafter can quickly generate high-quality 3D game assets, including characters, items, and scenes. Developers can obtain a large number of 3D models in a short time using a single image or text description, accelerating the game development process.
• Architecture and Interior DesignIn the fields of architecture and interior design, PartCrafter can be used to quickly generate architectural models and interior design schemes. Designers can upload architectural drawings or design sketches to quickly generate 3D models, visually presenting the schemes and helping clients better understand the design intent.
• Film and television productionPartCrafter's ultra-high-definition geometric detail modeling capabilities and multi-view input to generate PBR models can provide film and television production teams with high-quality 3D model materials.
• educateIn the field of education, PartCrafter can be used to demonstrate complex scientific concepts, such as molecular structure and human anatomy, to enhance students' understanding and learning outcomes.
• Augmented Reality/Virtual RealityPartCrafter generates realistic 3D models that help build more authentic AR/VR experiences. 3D models generated from single images can be directly used to construct virtual scenes, providing users with a more immersive experience.