PrimitiveAnything - A new 3D shape generation framework jointly launched by Tencent and Tsinghua University

What is PrimitiveAnything?

PrimitiveAnything is a novel 3D shape generation framework jointly developed by Tencent's AIPD (AI Platform Department) and Tsinghua University. It decomposes complex 3D shapes into simple primitive elements, generates these primitives through an autoregressive approach, and finally reassembles them into complete 3D shapes. The framework's core advantages lie in its high-quality generation results, powerful generalization ability, and high efficiency.

PrimitiveAnything's main functions

• High-quality 3D primitive assembly generationIt can generate high-quality 3D primitive assemblies that are geometrically faithful to the original model and conform to human intuitive understanding of shape.
• Diverse 3D content creationIt supports generating 3D content from text or image conditions, providing users with a flexible way to create content.
• Efficient storage and editingBecause primitive representations are used, the generated 3D models are more efficient in storage and easier to edit and adjust.
• Autoregressive Transformer ArchitectureIt generates 3D primitives frame by frame using an autoregressive transformer, which can handle primitive sequences of different lengths and can be easily extended to new primitive types.
• Unambiguous parameterization schemeBy eliminating ambiguities in parameterization, we ensure that the model maintains stability and accuracy during training and generation.
• Geometric fidelity and semantic consistencyDuring the generation process, PrimitiveAnything maintains high geometric fidelity and can generate semantic decomposition results that conform to human cognition.
• Modular designThe framework's modular design supports seamless integration of new primitive types without modifying the architecture, and can adapt to different primitive representations.

The technical principles of PrimitiveAnything

• Unambiguous parameterization scheme
  • Unified representation: Multiple types of primitives (such as cubes, elliptical cylinders, and ellipsoids) are used to represent 3D shapes under a unified parameterization scheme. The type, position, rotation, and scaling attributes of each primitive are encoded and input into the model.
  • Eliminate ambiguityTo address the inherent ambiguity in parameterization (e.g., different combinations of parameters may produce the same shape), the research team developed a comprehensive set of rules. By analyzing the symmetry of primitives, they selected the set of rotation parameters with the smallest L1 norm as the unique representation, ensuring the stability and accuracy of the training process.
• Autoregressive Transformer Architecture
  • Shape ConditioningThe framework employs a decoder-based Transformer architecture, capable of generating variable-length primitive sequences based on shape features. First, a point cloud encoder extracts feature representations of the 3D shape. Then, an autoregressive Transformer model receives these shape features and previously generated primitive information to predict the features of the next primitive.
  • Cascaded decoderTo model the dependencies between primitive attributes, the framework uses a cascaded decoder to predict the primitive's type, position, rotation, and scaling attributes sequentially. This captures the natural correlations between primitive attributes; for example, the primitive type influences its possible position, rotation, and scaling parameters, consistent with human assembly logic.
• Autoregressive generation process
  • Sequence generationThe entire primitive abstraction process is reimagined as a sequence generation task. The model takes point clouds as input and then generates primitive sequences autoregressively until the end marker is predicted.
  • Training objectivesDuring training, cross-entropy loss, Chamfer distance (for reconstruction accuracy), and Gumbel-Softmax (for differentiability) are combined until the end-of-sequence marker is generated. This allows for flexible and human-like decomposition of complex 3D shapes.

PrimitiveAnything's project address

• Project official website:https://primitiveanything.github.io/
• Github repository:https://github.com/PrimitiveAnything/PrimitiveAnything
• HuggingFace model library:https://huggingface.co/hyz317/PrimitiveAnything
• arXiv technical paper:https://arxiv.org/pdf/2505.04622

Application scenarios of PrimitiveAnything

• 3D Modeling and DesignPrimitiveAnything can quickly generate the "geometric skeleton" of complex 3D models, allowing designers to focus on refining details and improving work efficiency and design speed.
• Game asset generationGame designers can use PrimitiveAnything to quickly generate various game scenes and character models, improving development efficiency. Players can also create new characters or items by simply piecing together geometric blocks, which are automatically optimized and adjusted by AI and seamlessly integrated into the physics engine.
• User-generated content (UGC)The framework supports generating 3D content from text or image input, and users can easily edit the generated results, providing new possibilities for UGC in games.
• Virtual Reality (VR) and Augmented Reality (AR)PrimitiveAnything can quickly generate realistic 3D objects in virtual reality and augmented reality environments, enhancing the user experience.