DeepEyesV2 - Xiaohongshu's open-source multimodal intelligent agent model

What is DeepEyesV2?

DeepEyesV2, developed by the Xiaohongshu team, is a multimodal intelligent agent model that achieves powerful tool invocation and multimodal reasoning capabilities through a two-stage training method. The model can understand text and image information, supports proactively invoking external tools such as code execution and web searches, and integrates the results returned by these tools into the reasoning process to solve complex real-world problems. The first stage of the model uses supervised fine-tuning to establish a basic understanding of tool usage, while the second stage uses reinforcement learning to improve the model's tool invocation efficiency and generalization ability. DeepEyesV2 performs exceptionally well in the newly proposed RealX-Bench benchmark test, demonstrating powerful multi-skill coordination capabilities.

Main functions of DeepEyesV2

• Multimodal understandingThe model can process both text and image information simultaneously and understand complex text and image content.
• Active tool invocationActively invoke external tools, such as code execution environments and web searches, when needed to obtain additional information or perform complex tasks.
• Dynamic Reasoning and Decision MakingIntegrate the results of tool calls into the reasoning process and solve problems step by step through iteration.
• Task AdaptiveIt intelligently selects appropriate tools based on different task types (such as perception, reasoning, etc.) to improve efficiency and accuracy.
• Complex task solvingBy combining tools and iterative reasoning, we can solve complex tasks that require the coordination of multiple capabilities, such as integrated tasks that combine perception, search, and reasoning.

DeepEyesV2 Technical Principles

• Cold Start Phase:
  • Supervised Fine-tuning (SFT)Fine-tuning the model by using a large amount of sample data containing tool usage steps (such as perceptual, reasoning, and long thought chain data) helps the model build a basic understanding of tool usage.
  • Data DesignThe data covers a variety of task types, ensuring that the model can learn appropriate tool invocation strategies in different scenarios.
• Reinforcement Learning Phase:
  • Strategy optimizationBuilding upon the cold start, reinforcement learning is used to further optimize the model's tool invocation strategy, thereby improving efficiency and generalization ability.
  • TargetReduce unnecessary tool calls, and creatively combine tools in unseen complex scenarios to improve the flexibility and adaptability of the model.

DeepEyesV2 project address

• Project official websitehttps://visual-agent.github.io/
• GitHub repositoryhttps://github.com/Visual-Agent/DeepEyesV2
• arXiv technical paper: https://arxiv.org/pdf/2511.05271

Application scenarios of DeepEyesV2

• Intelligent question answering and information retrievalWhen users upload images to ask questions, DeepEyesV2 can combine image recognition and web search to provide accurate answers.
• Education and learning supportIt provides students with homework tutoring and knowledge exploration support through image recognition and reasoning.
• Content creation and editingAnalyze image content and provide image editing suggestions and related text generation.
• Intelligent customer service and technical supportUsing image recognition and web search, we provide users with troubleshooting and consultation solutions.
• HealthcareIt assists doctors in analyzing medical images and provides health consultations and preliminary diagnoses by combining them with online searches.