Unlimited-OCR - Baidu's open-source end-to-end long document OCR model

What is Unlimited-OCR?

Unlimited-OCR is an end-to-end long document OCR model launched by Baidu. It uses a Reference Sliding Window Attention mechanism to compress the decoder's key-value cache from linear growth to a constant, enabling the transcription of dozens of pages of documents in a single forward pass. Based on a 3B total parameter MoE architecture, the model achieved state-of-the-art (SOTA) performance on OmniDocBench v1.6 with a total score of 93.92% and an inference speed of 5580 TPS. The code and weights are fully open-source.

Main functions of Unlimited-OCR

• Long document recognition in one goSupports single forward transcription of PDF documents ranging from 2 pages to 40+ pages, without requiring page-by-page for loop processing.
• Multi-type document parsingIt covers 9 types of formats, including PPT, academic papers, books, color textbooks, test papers, magazines, newspapers, notes, and research reports.
• High-precision content extractionThe entire process, from text and formulas to tables, is output end-to-end in the order of reading, with a formula CDM accuracy of 95.79% and a table TEDS accuracy of 93.32%.
• Dual-resolution visual codingBase mode (1024×1024) is used for long documents with multiple pages, while Gundam mode (dynamic resolution) is used for high-precision recognition on a single page.
• Constant Delayed InferenceRegardless of the length of the output sequence, the KV cache remains constant at m+n, and the inference latency and memory usage remain stable.

The technical principle of Unlimited-OCR

• R-SWA attention mechanismEach generated token only considers all reference tokens (visual + cue words) and the most recent 128 output tokens. Visual tokens are excluded from the sliding window state transition to avoid visual features being gradually blurred during long-term generation.
• Constant KV Cache DesignThe KV cache is implemented as a queue with a capacity of m+n. Each time a new token is generated, the (m+1)th token is evicted. The computational cost and memory usage do not increase with the sequence length.
• DeepEncoder Visual EncodingIt adopts the SAM-ViT cascaded CLIP-ViT architecture, and performs 16 times token compression through the bridge layer. A 1024×1024 image is compressed into 256 visual tokens, which are then frozen after encoding once.
• MoE-LLM decoderThe MoE architecture features 3B total parameters and 500M activations, with all attention layers replaced by R-SWA. It is trained for 4000 steps based on DeepSeek-OCR checkpoint, with a global batch size of 256 and a maximum sequence size of 32K.
• Inference engine optimizationBoth Transformers and SGLang implement constant TPS and constant memory KV cache management, and the per-call latency is always low under the Flash Attention v3 kernel.

Follow us on WeChat and reply with "open source",join inAI open source project discussion group

How to use Unlimited-OCR

• Model Downloadvia Hugging Face baidu/Unlimited-OCR Or GitHub baidu/Unlimited-OCR Get the code and weight.
• Environmental preparationSupports the Transformers library and SGLang inference engine; requires a corresponding GPU environment.
• Input formatSupports PDF page image input; Base mode is used for multi-page long documents, and Gundam mode is used for single-page high-resolution recognition.
• Inference callThe entire document can be transcribed using OCR in a single forward pass, without the need for an external scheduler for pagination.
• Extended ApplicationsThe R-SWA mechanism can be migrated to long-output tasks such as ASR, translation, and subtitle generation.

The core advantages of Unlimited-OCR

• SOTA recognition accuracyOmniDocBench v1.5 achieved a total score of 93.23%, and v1.6 achieved a total score of 93.92%, ranking first in end-to-end performance.
• Constant resource usageKV cache does not increase with the number of document pages; the memory usage and latency remain stable for documents with 20 pages or 40+ pages.
• Speed increases with lengthThe longer the output, the more obvious the advantage. With 6144 tokens, the theoretical TPS limit is about 35% higher than DeepSeek-OCR.
• General decoding architectureR-SWA is a non-OCR-specific trick that is applicable to any generation task with a "reference source + long output".
• Lightweight open source:3B general parameters, 500M activation, the model and code are open source, which is convenient for deployment and secondary development.

Unlimited-OCR project address

• GitHub repositoryhttps://github.com/baidu/Unlimited-OCR
• HuggingFace model libraryhttps://github.com/baidu/Unlimited-OCR

Comparison of Unlimited-OCR with similar products

Application scenarios of Unlimited-OCR

• Enterprise record digitizationBatch processing of hundreds or thousands of scanned PDFs, ancient books, and bound volumes, allowing for structured extraction in one go without splitting them.
• Analysis of academic literatureEnd-to-end transcription of entire papers, journal collections, and research reports, preserving formulas, tables, and reading order.
• Educational test paper gradingBatch recognition of multi-page test papers and workbooks, supporting color textbooks and complex layouts.
• Legal contract reviewAccurate OCR extraction of lengthy contract texts for subsequent NLP analysis and compliance review.
• Multilingual translation pipelineAs a verification scenario for the general R-SWA decoding scheme, it can be extended to long-sequence tasks such as ASR and subtitle generation.