MM-Eureka - A multimodal inference model developed by Shanghai AI Lab in collaboration with Shanghai Jiao Tong University and other institutions.
MM-Eureka is a multimodal reasoning model jointly developed by researchers from the Shanghai Artificial Intelligence Laboratory, Shanghai Institute of Innovation, Shanghai Jiao Tong University, and the University of Hong Kong. The model utilizes rule-based large-scale reinforcement learning (RL) to...
What is MM-Eureka?
MM-Eureka is a multimodal reasoning model jointly developed by researchers from the Shanghai Artificial Intelligence Laboratory, Shanghai Institute of Innovation, Shanghai Jiao Tong University, and the University of Hong Kong. The model extends key features of unimodal reasoning (such as stable response length growth, accuracy rewards, and visual insight moments) to multimodal scenarios through rule-based large-scale reinforcement learning (RL).
MM-Eureka introduces two core models: MM-Eureka-8B and MM-Eureka-Zero-38B, based on InternVL2.5-Instruct-8B and InternVL2.5-Pretrained-38B, respectively. Using only 54K graph-text data for rule-based reinforcement learning training, its average performance surpasses that of the MPO model using 1M of data. MM-Eureka-Zero-38B, using only 8K graph-text mathematical reasoning data, outperforms the instruction model by 8.2% on its self-built K12 benchmark and performs similarly on MathVerse.
Main functions of MM-Eureka
- Multimodal reasoning abilityIt extends large-scale rule-based reinforcement learning (RL) to the field of multimodal reasoning, and can handle textual and visual information.
- Reproduce key featuresKey features of text RL systems such as DeepSeek-R1 are reproduced in a multimodal space, including accuracy rewards and steady improvement in response length, as well as the emergence of reflective behavior.
- Data efficiencyUsing only 54K of image and text data for regular RL training, the average performance outperformed the MPO model using 1M of data, and the overall benchmark accuracy was comparable to the model trained with CoT SFT using 12M of data.
MM-Eureka's technical principles
- Rule-based large-scale reinforcement learning frameworkMM-Eureka is an efficient and scalable multimodal large-scale reinforcement learning framework developed based on OpenRLHF, supporting various models and RL algorithms such as InternVL. It enables effective training of models in multimodal environments and successfully replicates the key features of DeepSeek-R1, such as a steady improvement in accuracy reward and response length.
- Data filtering and stable trainingThe research team discovered that data selection is crucial for stable RL training. A difficulty-based data filtering strategy plays a key role in the stability of RL training.
- Visual aha-momentMM-Eureka exhibits visual insight moments similar to DeepSeek-R1 during training. Specifically, the model learns to reflect and backtrack, re-examining key information in images.
- Minimalist reinforcement learning designThe minimalist RL design proved effective in MM-Eureka. When experimenting on the instructive model, adding KL divergence limited model exploration, resulting in unobservable improvements in response length. MM-Eureka employs simple reward functions (such as accuracy and format rewards) and achieves stable training through a difficulty-based data filtering strategy.
- Efficient data utilizationMM-Eureka demonstrates extremely high data efficiency. Using only 54K of graph-text data for rule-based reinforcement learning training, its average performance surpasses that of the MPO model using 1M of data. MM-Eureka-Zero, using only 8K of graph-text mathematical reasoning data (0.05% of the instruction model), outperforms the instruction model by 8.2% on the self-built K12 benchmark and performs similarly on MathVerse. This indicates that in the field of multimodal reasoning, simple rule-based reinforcement learning designs can significantly improve training effectiveness, achieving performance comparable to large-scale training even with limited data.
MM-Eureka project address
- Github repository:https://github.com/ModalMinds/MM-EUREKA
- arXiv technical paper:https://arxiv.org/pdf/2503.07365
Application scenarios of MM-Eureka
- EducationMM-Eureka helps students better understand and solve complex mathematical problems through powerful reasoning and reflection mechanisms.
- Augmented Reality (AR) and Virtual Reality (VR)In AR and VR scenarios, by combining visual and linguistic information, MM-Eureka can provide users with a more immersive and interactive experience.
- Data analysis and decision supportMM-Eureka's multimodal reasoning capabilities give it a significant advantage in data analysis and decision support. It can handle complex text and image data, helping users extract key information from large amounts of data and make more informed decisions.
- Automation and intelligent assistantsMM-Eureka can serve as the core technology for intelligent assistants, providing users with a more intelligent and natural interactive experience.
- Games and entertainmentIn the gaming and entertainment industry, MM-Eureka's multimodal reasoning capabilities can be used to develop more intelligent non-player characters (NPCs) and interactive storylines.