Dolphin - ByteDance's open-source document parsing model
Dolphin is a lightweight and efficient document parsing model open-sourced by ByteDance. Based on a two-stage approach—parsing the structure first and then the content—the first stage generates a sequence of document layout elements, and the second stage uses these elements as anchor points...
What is Dolphin?
Dolphin is a lightweight and efficient document parsing model open-sourced by ByteDance. Based on a two-stage approach—parsing the structure first and then the content—the first stage generates a sequence of document layout elements, and the second stage uses these elements as anchors to parse the content in parallel. Dolphin performs exceptionally well on various document parsing tasks, outperforming models such as GPT-4.1 and Mistral-OCR. With only 322M parameters, Dolphin is small in size, fast, and supports parsing various document elements, including text, tables, and formulas. Dolphin's code and pre-trained models are publicly available for developers to use and study.
Dolphin's main functions
- Layout AnalysisIt identifies various elements in a document (such as headings, charts, tables, footnotes, etc.) and generates a sequence of elements in the natural reading order.
- Content extraction: Parses the entire document page into structured JSON or Markdown format for easier subsequent processing and display.
- Text paragraph analysisIt accurately identifies and extracts text content from documents, and supports multiple languages (such as Chinese and English).
- Formula recognitionSupports the recognition of complex formulas, including inline formulas and block formulas, and outputs them in LaTeX format.
- Table AnalysisIt supports parsing complex table structures, extracting cell content, and generating HTML-formatted tables.
- Lightweight architectureThe model has 322M parameters, is small in size, runs fast, and is suitable for use in resource-constrained environments.
- Supports multiple input formatsIt supports processing various types of document images, including academic papers, business reports, and technical documents.
- Diverse output formatsIt supports outputting parsed results in multiple formats such as JSON, Markdown, and HTML, making it easy to integrate with different systems.
Dolphin's technical principles
- Page-level layout analysisThe input document image is encoded using a Swing Transformer to extract visual features. A sequence of document elements is generated based on the decoder, each element containing its category (e.g., title, table, chart, etc.) and coordinate position. The goal of this stage is to generate structured layout information in a natural reading order.
- Element-level content analysisBased on the layout information generated in the first stage, a partial view of each element is cropped from the original image. Specific prompts are used to parse the content of each element in parallel. For example, tables are parsed using specific prompts for HTML format, while formulas and text paragraphs share prompts for LaTeX format. The decoder then generates the final parsed content based on the cropped element images and the prompts.
Dolphin's project address
- GitHub repository:https://github.com/bytedance/Dolphin
- HuggingFace model library:https://huggingface.co/ByteDance/Dolphin
- arXiv technical paper:https://arxiv.org/pdf/2505.14059
- Experience the demo online:http://115.190.42.15:8888/dolphin/
Application scenarios of Dolphin
- academic researchIt analyzes the text, formulas, and charts in academic papers, aiding in literature review and data analysis.
- Commercial OfficeExtract key information from business documents to facilitate contract review and report generation.
- EducationDigitize textbooks and test papers to support online learning and multilingual teaching.
- Technology Development: To parse technical documents and facilitate code management and technical communication.
- Everyday ApplicationsQuickly process daily documents and improve office efficiency.