Light-R1 - 360 Brain's open-source long-chain reasoning model

What is Light-R1?

Light-R1 is an open-source AI model from 360 AI, focusing on long-chain reasoning in mathematics, specifically Light-R1-32B. Based on Qwen2.5-32B-Instruct, the model was trained using 70,000 mathematical data points and a two-stage learning process (SFT+DPO), achieving performance superior to DeepSeek-R1-Distill-Qwen-32B from scratch. In the AIME24 test, Light-R1 scored 76.6 points, significantly higher than DeepSeek-R1-Distill's 72.6 points. The model has low training costs, requiring only 12 H800 machines running for 6 hours at approximately $1000. The model is fully open-source, including the model, dataset, training framework, and evaluation code, contributing to the development of the open-source community and providing a reference for low-cost training of specialized models.

Main functions of Light-R1

• Efficient Mathematical Problem SolvingIt can quickly and accurately solve complex mathematical problems, including but not limited to algebra, geometry, probability and other fields.
• Improved reasoning abilityPossesses strong logical reasoning ability and can handle problems with long thought processes.
• Generalization abilityIt demonstrates generalization ability in other areas (such as logical reasoning and language comprehension).
• Low-cost training and deploymentHigh performance at extremely low cost, suitable for users or enterprises with limited resources to quickly deploy and apply.

Light-R1's technical principles

• Basic Model and Starting PointThe model is developed based on Qwen2.5-32B-Instruct, achieving a performance improvement from zero to surpassing DeepSeek-R1-Distill.
• Course Learning:
  • SFT (Supervised Fine-Tuning)The data was categorized by difficulty level, and supervised fine-tuning was conducted in two phases. The first phase used 70,000 data points, and the second phase selected the 3,000 most challenging data points for further fine-tuning.
  • DPO (Direct Preference Optimization)Based on SFT, the output quality of the model is optimized by multiple sampling and the construction of preference pairs.
• Data processing and deduplicationThe training data comes from multiple open-source mathematical datasets (such as OpenR1-Math-220k, OpenThoughts-114k, etc.) and has undergone rigorous data deduplication to avoid the impact of test data leakage on model performance.
• Model fusionThe final Light-R1-32B is obtained by fusing the SFT stage 2, DPO, and another DPO version of the model. This further improves the model's performance and stability.
• Training framework and optimizationThe 360-LLaMA-Factory training framework supports sequence parallelism and efficient distributed training. Based on the optimized training process, Light-R1 can complete training in just 6 hours on 12 H800 machines.

Light-R1 project address

• GitHub repository:https://github.com/Qihoo360/Light-R1
• HuggingFace model library:https://huggingface.co/collections/qihoo360/light-r1

Application scenarios of Light-R1

• EducationAs a math learning tool, it helps students solve complex problems, provides solution steps and ideas, and is suitable for math competitions and daily learning.
• Scientific research and academicIt assists in mathematical research and interdisciplinary problem-solving, such as physical modeling and engineering optimization.
• Enterprise ApplicationsIt is used to solve complex problems such as data analysis, risk assessment, and supply chain optimization.
• Software integrationIt can be integrated into smart assistants and mathematical software to enhance reasoning and problem-solving capabilities.
• Open source and developersIt supports developers in customizing and extending models, and promotes the development of the open source community.