PaddleOCR-VL - Baidu PaddlePaddle's open-source multimodal document parsing model

What is PaddleOCR-VL?

PaddleOCR-VL is an open-source multimodal document parsing model from the Baidu PaddlePaddle team. With only 0.9B parameters, it is optimized for low-computing-power devices. It achieved a score of 92.6 in the internationally authoritative OmnidocBench V1.5, ranking first globally and surpassing mainstream models such as GPT-4o. The model employs a two-stage architecture: PP-DocLayoutV2 handles layout analysis, while PaddleOCR-VL-0.9B completes content recognition. It supports 109 languages and can accurately handle complex elements such as tables, formulas, and charts, outputting structured Markdown/JSON data. Its lightweight design makes it suitable for local deployment, especially for scenarios with high privacy requirements such as medical reports and ancient book recognition.

Main functions of PaddleOCR-VL

• Intelligent document structure parsingIt automatically recognizes elements such as text, tables, formulas, and charts and maintains the correct reading order.
• Multilingual supportIt covers 109 languages (including Chinese, English, Japanese, Korean, etc.).
• Lightweight and efficient deploymentIt is suitable for resource-constrained devices such as mobile phones and local servers.
• Multimodal understandingIt can handle mixed text and image scenarios. The model performed excellently in the OmniDocBench V1.5 international benchmark, and is particularly good at accurately recognizing special scenarios such as medical reports, vertical text in ancient books, and mathematical formulas. It can output structured JSON or Markdown format data.

PaddleOCR-VL Technical Principles

• Two-phase processing architecture:usePage layout detection first, content recognition second.The process:
  • Phase 1:pass PP-DocLayoutV2 The model performs layout analysis, locates semantic regions such as text, tables, and formulas, and predicts the human reading order (with an error of only 0.043).
  • Phase Two:Depend on PaddleOCR-VL-0.9B It performs fine-grained recognition on the located areas and outputs structured text, tables, formulas, and other content.
    It avoids the illusion and misalignment problems common in end-to-end models, and improves the stability of processing complex layouts.
• Multimodal fusion core architectureThe core model integrates three major components:
  • Visual encoder:use NaViT Dynamic Resolution EncoderIt adaptively processes document images of different sizes and resolutions while preserving detailed information.
  • Language ModelBased on lightweight ERNIE-4.5-0.3BIt provides powerful language understanding and generation capabilities.
  • Cross-modal alignment mechanismThe visual-language fusion module converts image features into structured text output.
• Dynamic resolution and lightweight designNaViT encoders supportDynamic resolution adjustmentIt adaptively allocates computing resources based on document complexity, balancing efficiency and accuracy. The overall model has only 0.9B parameters, can run efficiently on a CPU, and its inference speed is 14.2%~253.01% faster than similar models.
• Multi-task unified frameworkIt uses an instruction-driven mechanism to uniformly process the recognition of elements such as text, tables, formulas, and charts, eliminating the need to switch models for different tasks and significantly reducing deployment complexity.

PaddleOCR-VL project address

• Project official website: https://ernie.baidu.com/blog/zh/posts/paddleocr-vl/
• HuggingFace model libraryhttps://huggingface.co/PaddlePaddle/PaddleOCR-VL
• arXiv technical paper: https://arxiv.org/pdf/2510.14528
• Experience the demo onlinehttps://huggingface.co/spaces/PaddlePaddle/PaddleOCR-VL_Online_Demo
• Official test address: https://aistudio.baidu.com/application/detail/98365

Application scenarios of PaddleOCR-VL

• Large-scale document digitizationIt is suitable for batch conversion of paper archives, historical documents, contracts, etc. into editable electronic formats, and supports accurate parsing of multilingual and complex layouts (such as tables and formulas).
• Financial and Commercial Paper ProcessingAutomatically identifies key information (such as amount, date, and company name) in invoices, receipts, and bank documents, improving the efficiency of financial auditing and tax management.
• Digitalization of Academic Research and EducationIt analyzes text, formulas, and charts in academic papers and textbooks, supports knowledge extraction and structured organization, and is suitable for scientific research information management and the development of intelligent education tools.
• Multilingual globalized document processingSupports 109 languages (including Arabic, Russian, Japanese and other special writing systems), suitable for multinational corporations, translation platforms and multilingual file management.
• Localized deployment in privacy-sensitive scenariosBecause the model is lightweight (0.9B parameters), it can run on ordinary CPUs or edge devices, making it suitable for fields with high data security requirements, such as government and healthcare.
• Intelligent Knowledge Base and Retrieval SystemCombined with RAG technology, it converts scanned documents into structured data, enhancing enterprise knowledge management efficiency and retrieval accuracy.