HyperEyes - A parallel multimodal search intelligent agent launched by Xiaohongshu in collaboration with Cambridge.

What is HyperEyes?

HyperEyes is a parallel multimodal search agent jointly developed by Xiaohongshu and the University of Cambridge. It pioneered the UGS paradigm, fusing visual localization and retrieval into a single atomic action, enabling concurrent multi-entity search in a single round. HyperEyes employs a dual-granularity efficiency-aware reinforcement learning framework. In six benchmark tests, the 30B version outperformed the strongest open-source model of the same scale by 9.9% in accuracy, reducing the number of tool call rounds by 5.3 times, achieving Pareto optimality in both accuracy and efficiency.

HyperEyes' main functions

• Parallel multimodal searchIt can concurrently locate and retrieve multiple entities in an image within a single round, replacing the traditional serial cropping-search process.
• Unified Evidence-Based Search: The visual positioning bounding box is directly embedded into the retrieval action parameters, and multiple target bounding boxes are carried in a single function call.
• Progressive rejection sampling: Refine efficient trajectories under an increasing round budget and construct 30,000 zero-redundancy parallel seed data.
• Dual-granularity efficiency-aware RLTRACE dynamically tightens the trajectory-level efficiency scale, while OPD injects token-level correction signals into failed trajectories.
• IMEB Benchmark EvaluationReleased a multi-entity visual evaluation set of 300 instances, with CAS scoring to jointly quantify accuracy and search efficiency.
• Multi-tool collaborationIt integrates image and text search tools, supporting the concurrent acquisition of visual and textual evidence.

HyperEyes's technical principles

• UGS Action Space ReconstructionThe visual positioning is transformed from an independent preliminary step into an embedded parameter of the retrieval action, and the physical level opens up a single-round multi-target concurrent path.
• Parallel data synthesis pipelineMulti-entity queries are synthesized by stitching together multiple types of images. Multi-constraint intersection problems are constructed based on graph random walks, and shortcut solutions are eliminated.
• TRACE Dynamic Reference RewardsThe current optimal trajectory is used as a dynamic benchmark. Rewards are given only when the model is more efficient than the benchmark, and the benchmark is automatically tightened each round.
• OPD asymmetric strategy distillationWhen the trajectory is ultimately answered incorrectly, the 235B teacher model is activated to provide dense token-level supervision for the failed trajectory, protecting the efficient concurrent instinct.
• Joint optimization objectiveGRPO combines trajectory-level efficiency rewards with token-level distillation losses to simultaneously optimize the accuracy and efficiency of the policy network.
• CAS cost perception scoreThe formula Acc²×100/(N_tok+2N_tool+1) is used to unify accuracy, token consumption, and tool rounds into an efficiency metric.

How to use HyperEyes

• Obtain open source resourcesVisit the GitHub repository https://github.com/DeepExperience/HyperEyes.
• Prepare the basic modelDownload and deploy Qwen3-VL-30B or 235B as the visual language model backbone, ensuring that the GPU memory is sufficient for inference.
• Configure external search tools: Integrate with image search and text search APIs (such as Bing Image Search and Google Custom Search) as an external tool for the Agent to perform parallel UGS searches.
• Input multi-entity queryUpload a complex image containing multiple entities, input a natural language question, and HyperEyes will automatically perform a unified evidence-based search using UGS, locating and retrieving all targets in a single round of concurrent processing.
• View parallel search resultsThe model returns structured search results and the final answer, including visual and textual evidence for each entity.
• Evaluate search efficiencyThe accuracy rate, token consumption, and overall efficiency of tool call rounds are quantitatively evaluated using the CAS (Cost Perceived Score) metric.

HyperEyes' core advantages

• Leap in efficiencyVersion 30B has an average of only 2.2 tool call rounds, which is 1/5 of the strongest open source model of the same scale, achieving a 5.3x efficiency improvement.
• Leading in accuracyIt outperforms the strongest open-source model of the same size by 9.9% across 6 benchmarks, and the 235B version is close to Gemini-3.1-Pro by 1.1%.
• Noise resistanceParallel strategies avoid the illusion of over-retrieval, improving accuracy by 3.7%-5.8% in mixed true and false evidence tests.
• Eliminate error cascadingUGS eliminates serial cropping dependencies, and pre-positioning deviations no longer pollute subsequent search results.
• Pareto optimalIt outperforms the joint frontier of accuracy and efficiency, with a CAS score 7.6 times that of the second-best open-source model.
• Full-stack efficiency reshapingFrom action space and data synthesis to RL training, the underlying structure is reconstructed to systematically solve the serial problem.

HyperEyes project address

• GitHub repositoryhttps://github.com/DeepExperience/HyperEyes
• arXiv technical paperhttps://arxiv.org/abs/2605.07177

HyperEyes Competitive Product Comparison

Application scenarios of HyperEyes

• Multi-character visual reasoningIt identifies the identity, profession, and historical events of specific individuals in a group photo and retrieves biographical information for multiple people.
• E-commerce product comparison: Perform concurrent searches on complex scene graphs containing multiple products to obtain price, brand, and review information.
• Cross-modal knowledge Q&A: Answer complex queries involving the relationships between multiple objects, landmarks, and artworks in an image.
• News Fact CheckConcurrent retrieval of news images containing multiple people and scenes to verify the authenticity of events and background information.
• Academic Chart AnalysisParallel location and content retrieval of screenshots of papers containing multiple charts, formulas, and citations.
• Social media moderationIt can perform concurrent searches on multiple entities in complex images to quickly identify illegal or misleading content.