Phi-4-reasoning-vision-15B - Microsoft's open-source multimodal inference model

What is Phi-4-reasoning-vision-15B?

Phi-4-reasoning-vision-15B is an open-source, 15 billion-parameter multimodal inference model from Microsoft, built upon the Phi-4-Reasoning language model and the SigLIP-2 visual encoder. Employing a "hybrid inference" design, the model automatically determines when deep thinking (mathematics/science problems) or rapid response (OCR/description tasks) is needed. It achieves a balance between accuracy and efficiency with only 200 billion tokens of training data. The model excels particularly in GUI understanding, mathematical reasoning, and document analysis, and its inference speed is more than 10 times faster than similar large-scale models.

Main functions of Phi-4-reasoning-vision-15B

• General Visual UnderstandingThe model can analyze image content to generate descriptions, answer questions about images, and identify objects, scenes, and text information.
• Document and Chart AnalysisThe model can read and understand various documents, receipts, invoices, and complex charts, extracting key information for quantitative analysis.
• Mathematical and Scientific ReasoningThe model excels at solving math and science problems that include formulas and diagrams, and can recognize handwritten content to display the complete solution process.
• GUI Interaction and Screen UnderstandingThe model can recognize interactive elements in computer and mobile phone interfaces, provide accurate bounding box coordinates, and support automated operation agents.
• Multi-image sequence analysisThe model can process multiple images, analyze time series changes, and understand the correlation and evolution trend between images.
• Adaptive Inference ModeThe system automatically selects between direct response and in-depth thinking based on the complexity of the task. Users can force the user to specify the reasoning method through special markers.

Technical principles of Phi-4-reasoning-vision-15B

• Architecture DesignIt adopts a mid-fusion architecture, uses a SigLIP-2 Naflex dynamic resolution visual encoder to process image input, generates visual tokens, and projects them into the embedding space of the Phi-4-Reasoning language model to achieve cross-modal joint reasoning.
• Hybrid reasoning mechanismThe model inherits from a language model backbone with reasoning capabilities. Through supervised fine-tuning, it learns to distinguish between tasks requiring reasoning (mathematics/science) and perceptual tasks (OCR/description). The training data is matched with 20% reasoning samples and 80% non-reasoning samples, enabling the model to adaptively select the reasoning depth.
• High-resolution processingThe model uses the SigLIP-2 Naflex variant that supports dynamic resolution, with up to 3600 visual tokens, equivalent to native 720p resolution, significantly improving the perception accuracy for high-density information interfaces and small interactive elements.
• Data StrategyThe model primarily uses rigorously selected open-source data, with quality improved through manual review, error correction, and synthetic generation. It is supplemented by domain-specific mathematical and GUI datasets, achieving performance comparable to models with trillions of tokens using only 200 billion tokens.

Project address for Phi-4-reasoning-vision-15B

• Project official website: https://www.microsoft.com/en-us/research/blog/phi-4-reasoning-vision-and-the-lessons-of-training-a-multimodal-reasoning-model/
• GitHub repositoryhttps://github.com/microsoft/phi-4-reasoning-vision-15B
• HuggingFace model libraryhttps://huggingface.co/microsoft/Phi-4-reasoning-vision-15B
• Technical Papers: https://www.microsoft.com/en-us/research/wp-content/uploads/2026/03/Phi-4-reasoning-vision-15B-Tech-Report.pdf

Application scenarios of Phi-4-reasoning-vision-15B

• Educational SupportStudents photograph math homework or physics diagrams, and the model recognizes the handwritten content, checks for errors in the solution steps, and provides the correct derivation process.
• Smart OfficeThe model supports automatic parsing of invoices and receipts, extracting information such as amount, date, and item, performing accounting calculations, and generating structured data output.
• Interface AutomationAs a basic model for computer use agents, it identifies buttons, input boxes, and menu elements on the screen to enable cross-platform automated operations and workflow execution.
• Mobile device assistanceThe model supports understanding mobile application interfaces, helping users locate function entry points, fill out forms, or perform complex multi-step tasks.
• Document AnalysisThe model can process scanned PDFs, charts, and reports, performing information extraction, trend analysis, and cross-document content comparison.