LaDeCo - An automated graphic design and composition method jointly developed by Xi'an Jiaotong University and Microsoft.
LaDeCo is an automated graphic design and composition method jointly developed by Xi'an Jiaotong University and Microsoft Research. It's based on breaking down design tasks into hierarchical steps. LaDeCo performs layer planning on the input design elements, dividing them into...
What is LaDeCo?
LaDeCo is an automated graphic design composition method jointly developed by Xi'an Jiaotong University and Microsoft Research. It is based on breaking down design tasks into hierarchical steps. LaDeCo performs layer planning on the input design elements, assigning them to different semantic layers, such as background, bottom layer, image/logo, text, and decoration. Then, LaDeCo predicts layer by layer, generating element attributes for each design layer, using the rendered images of previously generated layers as contextual information to guide the generation of subsequent layers. LaDeCo processes multimodal input based on large multimodal models (LMMs), supporting design subtasks that do not require task-specific training, such as resolution adjustment, element filling, and design variations.
LaDeCo's main functions
- Layer planningAutomatically assigns input multimodal design elements (such as images and text) to different semantic layers, such as background, bottom layer, image/logo, text, and decoration layers.
- Hierarchical design generationBased on the results of layer planning, the element attributes of each layer are predicted and generated step by step to create a complete design diagram.
- Resolution adjustmentAdjust the design according to different canvas sizes to make the design attractive on canvases of different sizes.
- Element FillAdding new elements to an existing design enhances its appeal.
- Design ChangesGiven the same input elements, create a variety of different designs to provide users with multiple choices.
LaDeCo's technical principles
- Large-scale multimodal models (LMMs)Based on a large-scale multimodal model, it understands the multimodal context and generates cross-domain responses.
- Layer planning moduleBased on pre-trained LMMs (e.g., GPT-4o), the semantic labels of input elements are predicted, enabling automatic classification of elements to the design layer.
- Hierarchical generation processBased on the results of layer planning, the attributes of design elements are generated layer by layer, and the rendered images of the generated layers are fed back to the model as context information to guide the generation of subsequent layers.
- Visual encoder and projectorUsed to encode element images and intermediate designs, generate image embeddings, and project them to match the hidden state dimensions required by the LMMs backbone.
- Chain-of-Thought ReasoningLaDeCo's hierarchical generation method embodies the concept of chain-thinking reasoning, improving reasoning performance by progressively generating and adjusting design layers.
LaDeCo's project address
- Project official website:elements2design.github.io
- arXiv technical paper:https://arxiv.org/pdf/2412.19712
LaDeCo Application Scenarios
- DesignerIt helps designers automatically complete graphic design and composition tasks, improving design efficiency and quality.
- Researchers and plannersIn landscape change studies, aesthetic assessments, and visual impact assessments, this technology enables researchers and planners to quickly and objectively calculate the proportion of visual elements in images, simplifying the assessment process.
- assessorsIt plays an important role in assessing visual landscape elements, helping assessors to conduct more efficient visual element analysis.
- DevelopersFor developers, LaDeCo is used to develop different applications.
- younger demographicLaDeCo's application in the field of automated graphic design attracts people aged 19-35 who have a high preference for creative content, sharing, music, short videos, games, and fashion.