Mistral OCR 4 - Mistral AI's latest generation document understanding model

What is Mistral OCR 4?

Mistral OCR 4 is the latest generation document understanding model from Mistral AI. The model supports extracting text from complex documents such as PDFs, images, and presentations, and returns structured output with bounding box localization, region type classification, and confidence scores. It supports 170 languages and is designed for downstream scenarios such as RAGs, agent workflows, and enterprise search.

Main functions of Mistral OCR 4

• High-precision document parsingSupports common enterprise formats such as PDF, DOC, PPT, and OpenDocument, and extracts rich elements such as text, tables, formulas, images, and signatures.
• Structured outputEach content block includes border coordinates, type tags such as title, table, formula, signature, etc., and page-by-page/word-by-word confidence scores.
• Multilingual supportIt covers 170 languages across 10 language families, with a particularly strong performance in low-resource languages.
• Dual-mode outputThe same endpoint supports pure extraction mode, Markdown + structured metadata and Document AI mode (outputting structured JSON according to a custom schema).
• Ultra-high throughputA single node can process up to 2,000 pages of documents per minute, making it suitable for large-scale batch processing scenarios.

Technical principles of Mistral OCR 4

• Visual text detection and classificationBased on the CNN/Transformer architecture, text regions in a document are located, and each region is semantically classified to restore the hierarchical structure of the original document.
• Sequence-to-sequence text recognitionThe detected character stream is converted into editable text using Seq2Seq or CTC models, and image preprocessing is combined to improve recognition accuracy.
• Structured semantic chunkingIt segments documents into semantic blocks with type labels and coordinates, providing downstream RAG systems with reference-ready units that can be directly used for retrieval, and supporting agents to perform structured operations.
• Single-container lightweight deploymentThe model is compact enough to enable self-hosted deployment within a single container, meeting data residency and compliance requirements.

How to use Mistral OCR 4

• Register to get API KeyVisit the Mistral AI Developer Platform La Plateforme, register an account, and then go to the "API Keys" page to create and save a key.
• Install SDKExecute in a Python environment pip install mistralai Install the official SDK.
• Call the API to process documents use client.ocr.process() The method takes a document URL or a local file as input and sets... model="mistral-ocr-latest"and turn on include_blocks=True To obtain border and type information, you can also... confidence_scores_granularity="word" Obtain word-by-word confidence.
• Parse the structured resultsAPI returns information containing... pages The JSON of the array, containing per page markdown text,images,tables,hyperlinks,dimensions and confidence_scores Fields such as these can be directly integrated into RAG or agent workflows.
• Batch processing to reduce costsFor high-throughput scenarios, it is recommended to submit tasks through the Batch Inference API to enjoy a 50% price discount.