DeepSeek-OCR - An open-source visual language model from the DeepSeek team.

What is DeepSeek-OCR?

DeepSeek-OCR is a visual language model developed by the DeepSeek team, focusing on efficiently processing long text content through optical compression technology. The model consists of a DeepEncoder encoder and a DeepSeek3B-MoE decoder, significantly reducing activation memory and the number of visual tags while maintaining high-resolution input. The model achieves 97% OCR accuracy at a 10x compression ratio and maintains 60% accuracy at a 20x compression ratio. DeepSeek-OCR supports multiple resolution modes, is suitable for multilingual document processing, and can parse complex content such as charts and chemical formulas, providing an efficient solution for large-scale document processing.

Main functions of DeepSeek-OCR

• Visual text compressionIt can efficiently compress long text content through visual modalities, achieving a compression ratio of 7-20 times.
• Multilingual OCRIt supports document recognition in nearly 100 languages, including Chinese, English, Arabic, Sinhala, etc.
• In-depth analysisIt can analyze complex content such as charts, chemical formulas, and geometric figures.
• Multi-format outputSupports Markdown format with layout and free OCR format without layout.

The technical principle of DeepSeek-OCR

• Core ArchitectureDeepSeek-OCR consists of two parts:
  • DeepEncoder: Responsible for encoding the input image (document) into a visual token.
  • DeepSeek-3B-MoE-A570MAs a decoder, it is responsible for decoding visual tokens into text.
• DeepEncoderDeepEncoder is the core component of DeepSeek-OCR, designed to maintain low active memory usage while achieving a high compression ratio for high-resolution input. It consists of the following parts:
  • Twin Towers:
    • SAM-base (80M)Based on window attention, it is mainly used to perceive local features and has low memory usage when processing high-resolution input.
    • CLIP-large (300M)This is based on global attention and is used to extract global semantic information. Because the input is compressed, the memory usage of the global attention component can be effectively controlled.
  • 16× Convolutional Compressed LayerBetween SAM and CLIP, DeepEncoder uses a 16× convolutional compression module. This module reduces the number of visual tokens from 4096 to 256 using two convolutional layers (each with a stride of 2). This design significantly reduces the number of visual tokens and lowers memory usage without losing important information.
  • Multi-resolution supportDeepEncoder supports multiple resolution modes, including Tiny, Small, Base, Large, and Gundam. Each mode corresponds to a different input resolution and number of visual tokens. For example:
    • Tiny512×512 resolution, output 64 visual tokens.
    • Small: 640×640 resolution, output 100 visual tokens.
    • Base: 1024×1024 resolution, output 256 visual tokens.
    • Large1280×1280 resolution, output 400 visual tokens.
    • GundamDynamic resolution supports higher resolution input and further reduces active memory through chunked processing.
• Decoder: DeepSeek-3B-MoE-A570MThe decoder, based on the DeepSeek-3B-MoE architecture, has 570M activation parameters. It is responsible for decoding the compressed visual tokens into text. The decoder converts the visual tokens into text representations through a non-linear mapping, specifically using the following formula: fdec​:Rn×dlatent​→RN×dtext​ in n It is the number of visual tokens.N It is the number of text tokens.dlatent​ and dtext​ These are the dimensions of visual tokens and text tokens, respectively.

DeepSeek-OCR project address

• GitHub repository: https://github.com/deepseek-ai/DeepSeek-OCR
• HuggingFace model libraryhttps://huggingface.co/deepseek-ai/DeepSeek-OCR
• Technical Papers: https://github.com/deepseek-ai/DeepSeek-OCR/blob/main/DeepSeek_OCR_paper.pdf

Application scenarios of DeepSeek-OCR

• Large-scale training data generationIt automatically processes hundreds of thousands of pages of documents every day, providing massive amounts of high-quality training data for large-scale language models and visual language models.
• Enterprise-level document digitizationIt can quickly and accurately convert various documents such as paper contracts and reports within an enterprise into searchable and editable digital formats.
• Academic research and document processingIt accurately parses complex content in academic papers, such as mathematical formulas, chemical formulas, and diagrams, and converts them into a structured, machine-readable format.
• Multilingual internationalization document processingEasily handle the multilingual document processing needs of multinational corporations or organizations in their global business operations.
• Financial and Business Intelligence AnalysisIt can deeply analyze the charts in research reports and convert them into structured data, providing automated support for financial analysis and investment decisions.