MonkeyOCR - A document parsing model jointly developed by Huazhong Technology and Kingsoft Office
MonkeyOCR is a document parsing model jointly developed by Huazhong University of Science and Technology and Kingsoft Office. The model supports the efficient conversion of unstructured document content into structured information. Based on accurate layout analysis, ...
What is MonkeyOCR?
MonkeyOCR is a document parsing model jointly developed by Huazhong University of Science and Technology and Kingsoft Office. The model efficiently converts unstructured document content into structured information. Based on precise layout analysis, content recognition, and logical sorting, it significantly improves the accuracy and efficiency of document parsing. Compared to traditional methods, MonkeyOCR performs exceptionally well when processing complex documents (such as those containing formulas and tables), with an average performance improvement of 5.1%, and improvements of 15.0% and 8.6% in formula and table parsing, respectively. The model excels in processing multi-page documents, achieving 0.84 pages per second, far exceeding other similar tools. MonkeyOCR supports various document types, including academic papers, textbooks, and newspapers, and is compatible with multiple languages, providing powerful support for document digitization and automated processing.
Main functions of MonkeyOCR
- Document parsing and structuringIt converts unstructured content (including text, tables, formulas, images, etc.) in various document formats (such as PDF, images, etc.) into structured machine-readable information.
- Multilingual supportIt supports multiple languages, including Chinese and English.
- Efficiently process complex documentsIt performs exceptionally well when handling complex documents, such as those containing formulas, tables, and multi-column layouts.
- Fast multi-page document processingIt efficiently processes multi-page documents, achieving a processing speed of 0.84 pages per second, significantly outperforming other tools (such as MinerU at 0.65 pages per second and Qwen2.5-VL-7B at 0.12 pages per second).
- Flexible deployment and expansionIt supports efficient deployment on a single NVIDIA 3090 GPU to meet the needs of different scales.
MonkeyOCR's technical principles
- Structure-Recognition-Relationship (SRR) Triple ParadigmThis document layout detector, based on YOLO, identifies the location and category of key elements (such as text blocks, tables, formulas, and images) in a document. It performs content recognition on each detected region, using a large multimodal model (LMM) for end-to-end recognition to ensure high accuracy. Based on a block-level reading order prediction mechanism, it determines the logical relationships between detected elements and reconstructs the semantic structure of the document.
- MonkeyDoc datasetMonkeyDoc is the most comprehensive document parsing dataset to date, containing 3.9 million instances covering more than ten document types in both Chinese and English. The dataset is built on a multi-stage pipeline, integrating meticulous manual annotation, procedural synthesis, and model-driven automatic annotation. It is used to train and evaluate MonkeyOCR models, ensuring strong generalization capabilities across diverse and complex document scenarios.
- Model optimization and deploymentThe AdamW optimizer and cosine learning rate scheduling are used for training on large-scale datasets to ensure a balance between accuracy and efficiency. Based on the LMDeploy tool, MonkeyOCR can run efficiently on a single NVIDIA 3090 GPU, supporting fast inference and large-scale deployment.
MonkeyOCR project address
- GitHub repository:https://github.com/Yuliang-Liu/MonkeyOCR
- HuggingFace model library:https://huggingface.co/echo840/MonkeyOCR
- arXiv technical paper:https://arxiv.org/pdf/2506.05218
- Experience the demo online:http://vlrlabmonkey.xyz:7685/
Application scenarios of MonkeyOCR
- Automated business processesFor internal enterprise document processing, such as contracts, reports, and invoices, it enables automatic data extraction and structuring, improving efficiency and reducing manual intervention.
- Digital ArchivesLibraries, archives, and other institutions digitize paper documents to facilitate long-term preservation and retrieval.
- Smart EducationEducational institutions analyze textbooks, test papers, academic papers, etc., and extract content for use in online learning platforms or teaching resource databases.
- Medical Record ManagementThe hospital analyzes medical documents such as medical records and examination reports to extract key information for use in the electronic medical record system, thereby improving data management efficiency.
- academic researchResearchers analyze a large number of academic documents to extract key information for literature reviews and data analysis, thus assisting in research work.