DeepSeek-OCR 2 - The second-generation OCR model open-sourced by the DeepSeek team.

What is DeepSeek-OCR 2?

DeepSeek-OCR 2 is the second-generation OCR model from the DeepSeek team. By introducing the DeepEncoder V2 architecture, it achieves a paradigm shift from fixed scanning to semantic reasoning. The model employs causal stream querying and a dual-stream attention mechanism, dynamically rearranging visual tokens to more accurately reconstruct the natural reading logic of complex documents. In the OmniDocBench v1.5 benchmark, the model achieved a comprehensive score of 91.09%, a significant improvement over its predecessor, while also significantly reducing the repetition rate of OCR recognition results, providing a new path for building a full-modal encoder in the future.

Main functions of DeepSeek-OCR 2

• Complex document parsingThe model can accurately parse complex documents with tables, formulas, and multi-column layouts, restoring the natural reading logic.
• High-efficiency visual compressionComplex document pages can be covered with only 256 to 1120 visual tokens, significantly reducing computational overhead.
• Dynamic semantic reorderingThe model uses causal flow queries to dynamically adjust the order of visual tokens based on image semantics, breaking the limitations of traditional fixed scanning.
• High-precision recognitionIn the OmniDocBench v1.5 benchmark, it achieved an overall score of 91.09%, a significant improvement over its predecessor, and performed exceptionally well in reading order recognition.

The technical principles of DeepSeek-OCR 2

• DeepEncoder V2 Architecture:The visual tokenizer discretizes images into visual tokens, employing SAM-base and two convolutional layers, with an output dimension of 896.We introduce causal flow queries, where visual tokens use bidirectional attention and causal flow queries use causal attention to achieve semantic reordering.
• Causal reasoning mechanism:By dynamically rearranging visual tokens through causal flow queries, the encoder can dynamically adjust the order of tokens based on image semantics.This mechanism is highly consistent with the unidirectional attention pattern of LLM, and can better fit continuous visual semantics.
• decoder:We continue to use the DeepSeek-MoE Decoder from DeepSeek-OCR, with a parameter scale of 3 billion, of which approximately 500 million parameters are activated during inference.
• Training process:It is divided into three stages: encoder pre-training, query augmentation, and decoder specialization. The model performance is improved through multi-stage optimization.

Project address for DeepSeek-OCR 2

• GitHub repository: https://github.com/deepseek-ai/DeepSeek-OCR-2
• HuggingFace model libraryhttps://huggingface.co/deepseek-ai/DeepSeek-OCR-2
• Technical Papers: https://github.com/deepseek-ai/DeepSeek-OCR-2/blob/main/DeepSeek_OCR2_paper.pdf

Application scenarios of DeepSeek-OCR 2

• Document processing and digitizationThe model can quickly convert paper documents into editable electronic documents, supports high-precision recognition of complex layouts and multilingual content, and is suitable for digital work in institutions such as libraries and archives.
• Academic and Scientific ResearchIt efficiently parses formulas, charts, and multi-column text in academic papers, helping researchers quickly extract key information and improve the efficiency of literature review and data analysis.
• Enterprise office automationThe model can automatically identify key information in documents such as contracts and reports, supporting rapid review, archiving, and retrieval of enterprise documents, thus improving office efficiency.
• EducationIt can quickly digitize documents such as textbooks and test papers, support online teaching and electronic examinations, assist teachers and students in organizing academic materials, and improve teaching and learning efficiency.
• Publishing and MediaThe model quickly parses the complex layouts of magazines and newspapers, supports electronic version production and content distribution, and helps the media industry achieve efficient content creation and management.