Vision2Web - A visual website development evaluation benchmark jointly launched by Tsinghua University and Zhipu AI

What is Vision2Web?

Vision2Web is a joint project launched by Tsinghua University and Zhipu AI.Visual Website Development Evaluation BenchmarkVision2Web is used to evaluate the end-to-end website building capabilities of multimodal AI agents. It includes 193 real-world website tasks, divided into three progressively more difficult levels: static webpage generation, interactive front-end development, and full-stack website construction. The benchmark employs an innovative "workflow-based agent verification" paradigm, combining GUI agent verification of functional correctness with VLM evaluation of visual fidelity, systematically revealing the capability boundaries of current state-of-the-art (SOTA) models in complex, long-term software development tasks.

Vision2Web's main functions

• Three-tiered progressive competency assessmentFrom static webpage generation to interactive front-end development, and then to full-stack website construction, the visual understanding and engineering implementation capabilities of the AI Agent are tested step by step.
• Two-dimensional automated verificationThe system verifies the correctness of functions through a GUI Agent and evaluates the visual fidelity using a VLM Judge, thus achieving objective and reproducible end-to-end testing.
• Real-world data supportThe dataset is built based on 193 real websites, covering four main categories and 16 subcategories, and provides 918 prototype diagrams and 1,255 test cases.
• Systemic capacity diagnosis: Accurately pinpoint the capabilities and failure reasons of agents in cross-modal reasoning, long-term planning, and complex system construction.

How to use Vision2Web

• Environmental preparationConfigure a containerized runtime environment that includes front-end, back-end, and database dependencies.
• Data loadingPlace prototype diagrams, requirements documents, and multimedia resources into the specified working directory.
• Agent access: Integrate the model under test into an agent framework such as OpenHands or Claude Code.
• Task executionThe Agent reads the input and generates code, ultimately outputting a deployable website project.
• Automated deploymentRun the startup script generated by the Agent to complete the website deployment on a fixed port.
• Functional verificationThe GUI Agent Verifier executes expert-designed test workflows, evaluates the correctness of interactions, and outputs a Functional Score.
• Visual verificationVLM Judge compares the rendered result with the prototype image, evaluates the visual fidelity, and outputs a Visual Score.
• Results AnalysisBy combining the scores from both dimensions, we can pinpoint the model's shortcomings in specific development stages.

Key information and usage requirements of Vision2Web

• Research BackgroundThis is a joint initiative by Tsinghua University and Zhipu AI, designed to systematically evaluate the visual website development capabilities of multimodal AI agents.
• Task sizeIt includes 193 real-world website tasks, divided into three progressively more difficult levels: static webpage generation, interactive front-end development, and full-stack website construction.
• Data compositionIt provides 918 multi-device prototype diagrams and 1,255 test cases, covering 16 subcategories across four major areas: content, transactions, SaaS, and public services.
• Verification mechanismA two-dimensional automated verification paradigm is adopted, which uses a GUI Agent to verify the correctness of the function (Functional Score) and a VLM Judge to evaluate the visual fidelity (Visual Score).
• Environmental requirementsRequires a containerized runtime environment with pre-installed front-end framework, back-end services, database, and other complete dependencies.
• Framework AccessThe model to be tested needs to be integrated into a standardized agent framework such as OpenHands or Claude Code.

Vision2Web's core advantages

• Task design is hierarchical and progressive.The three-tier architecture, from static web pages to full-stack development, can accurately pinpoint the capability boundaries of the Agent at different development stages.
• Data sourced from real-world scenarios: Based on the C4 validation set, a real website is built to avoid the bias of synthetic data and ensure that the evaluation results are close to the actual application.
• The verification mechanism is objective and reliable.The GUI Agent and VLM Judge components work together to achieve reproducible and automated evaluation of functional correctness and visual fidelity.
• Comprehensive and diverse coverageIt covers four major areas and 16 subcategories, and supports systematic comparative analysis across models, frameworks, and devices.
• Diagnostic capabilities are refined and in-depthBy using workflow-based verification nodes, the specific reasons for failures of the Agent in UI understanding, interaction logic, long-term planning, and other aspects can be clearly traced.

Vision2Web project address

• arXiv technical paper: https://arxiv.org/pdf/2603.26648

Comparison of Vision2Web with similar competitors

Comparison Dimensions	Vision2Web	Design2Code	WebGen Bench
Task Scope	Three-tiered progression: static web pages, interactive front-end, and full-stack development.	Static UI to Code Generation Only	Text-driven end-to-end development lacks multimodal support.
Verification method	GUI Agent + VLM Judge dual-component workflow verification	Pixel-level comparison and rule scripts	Open-ended text evaluation, lacking structured constraints
Reproducibility	High: Structured workflows clearly constrain execution paths	Low: Layout changes make the script vulnerable.	Low: Open-ended assessments are difficult to reproduce
Data scale	193 tasks / 918 prototype diagrams / 1255 test cases / 16 subclasses	Limited in scale and with limited category coverage	The task hierarchy is not clearly defined, and the number of tasks is relatively small.
Multi-device support	Supports adaptation evaluation for desktop/tablet/mobile devices.	The multi-device scenario was not clearly distinguished.	Multi-device compatibility not emphasized
Capability Diagnosis	Precise: Capable of pinpointing weaknesses in UI understanding, interaction logic, and system architecture.	Roughness: Visual similarity score only	Rough: Overall end-to-end rating, difficult to break down into smaller parts.

Application scenarios of Vision2Web

• Model capability assessmentProvides standardized benchmark tests for visual website development capabilities for multimodal large models such as Claude, Gemini, and GPT.
• Agent framework optimizationCompare the performance of different agent frameworks such as OpenHands and Claude Code to guide framework iteration and upgrades.
• Algorithm Development and Verification: Evaluate the innovative effects of the new model on key technologies such as UI understanding, code generation, and long-term planning.
• Product capability benchmarking: To help AI website building products (such as Vibe Coding tool) quantify their own level and clarify the gap with the state-of-the-art (SOTA) products.
• Education and Training ReferenceIt serves as a teaching case library to train engineers and researchers in AI-assisted development.