ScreenCoder - An open-source tool for generating front-end code from intelligent UI screenshots.

What is ScreenCoder?

ScreenCoder is an open-source intelligent UI screenshot-to-code system that supports quickly converting any design screenshot into clean, editable HTML/CSS code. ScreenCoder uses a modular multi-agent architecture, combining visual understanding, layout planning, and code synthesis technologies to generate high-precision, semantic front-end code. Users can easily modify the layout and styles according to their needs, achieving seamless integration between design and development. It is suitable for rapid prototyping and pixel-perfect interface construction, greatly improving front-end development efficiency.

ScreenCoder's main functions

• UI screenshot to codeIt supports quickly converting any UI screenshot or design prototype into clean, editable HTML/CSS code.
• High-precision code generationThe generated code is highly consistent with the original design, visually aligned, and faithfully reproduces the semantics.
• Custom modificationIt allows users to adjust the layout and style according to their needs, facilitating secondary development.
• Multi-model supportIt supports multiple generative models such as Doubao, Qwen, GPT, and Gemini, allowing users to choose according to their needs.
• Rapid deploymentThe generated code can be used directly in the production environment, supporting rapid prototyping and pixel-perfect interface construction.

ScreenCoder's technical principles

• Grounding Agent PhaseThe localization phase uses a Visual Language Model (VLM) to identify and label key structural components in the UI image, such as the sidebar, header, and navigation bar. Text prompts guide the model to detect specific components, returning their bounding boxes and semantic labels. To ensure the accuracy and reliability of the detection results, the system performs deduplication, conflict resolution, and rollback operations, and infers the main content area. Finally, a layout dictionary is output, providing foundational information for subsequent layout planning and code generation.
• Planning AgentDuring the planning phase, a hierarchical layout tree is constructed based on the output of the positioning phase, providing a structural context for code generation. Detected components are organized into a tree structure using simple spatial heuristics and composition rules. The system creates a root container to fill the viewport, generates absolutely positioned .box elements for each top-level area, and inserts inner `<div class="container grid">` elements as needed to implement the CSS Grid layout. Each node is annotated with grid template configuration and sorting metadata, facilitating direct compilation into HTML/CSS code.
• Generation Phase (Generation Agent)The generation phase transforms the semantic layout tree into executable HTML/CSS code. ScreenCoder uses a natural language-driven generation process to build adaptive hints for each component in the layout tree, generating corresponding code through a language model. The hints contain semantic tags for the components and layout context, with user instructions (if provided) appended to them. The generated code is assembled according to the structure of the layout tree, preserving hierarchy, order, and layout configuration. The system replaces gray placeholders in the generated code with actual images from the original screenshots, restoring visual and semantic consistency.

ScreenCoder's official website address

• GitHub repository: https://github.com/leigest519/ScreenCoder
• arXiv technical paper: https://arxiv.org/pdf/2507.22827
• Experience the demo onlinehttps://huggingface.co/spaces/Jimmyzheng-10/ScreenCoder

Application scenarios of ScreenCoder

• Accelerating front-end developmentQuickly convert UI design screenshots into high-quality HTML/CSS code, significantly shortening the front-end development cycle and helping development teams improve efficiency and reduce manual coding workload.
• Design and Development CollaborationThis feature directly converts design screenshots into actionable code, facilitating seamless collaboration between design and development teams, reducing communication costs, and ensuring the accurate communication of design intent.
• Rapid prototypingIt can instantly transform design concepts into interactive front-end prototypes, accelerating the early validation and user testing process of product design, and supporting rapid iteration and optimization of user experience.
• Education and TrainingAs an educational tool, it helps students and novice developers intuitively understand the relationship between UI design and front-end code, accelerating the learning process and improving practical skills.
• Small teams and startupsIt provides efficient code generation solutions for small teams and startups with limited resources, helping them quickly launch product prototypes or minimum viable products (MVPs), reduce development costs, and accelerate time to market.