rStar2-Agent - Microsoft's open-source mathematical inference model

What is rStar2-Agent?

rStar2-Agent is an open-source mathematical reasoning model from Microsoft with only 14 billion parameters. Trained using agent reinforcement learning, it achieves an accuracy of 80.6% in the AIME24 mathematical reasoning test, surpassing DeepSeek-R1 with its 671 billion parameters. The model excels in mathematical reasoning and demonstrates strong generalization capabilities in tasks such as scientific reasoning and agent tool usage. Through three major technological breakthroughs—efficient training infrastructure, innovative algorithms, and a unique training process—the model achieves high performance and low computational cost, bringing new ideas to the field of AI reasoning.

Main functions of rStar2-Agent

• Efficient Mathematical ReasoningIn mathematical reasoning tests such as AIME24, it achieved a high accuracy rate of 80.6% with 14 billion parameters, surpassing models with a much larger number of parameters, and can quickly and accurately solve complex mathematical problems.
• Scientific reasoning abilityIn the GPQA-Diamond scientific reasoning benchmark test, the accuracy rate reached 60.9%, demonstrating the ability to understand and reason about scientific knowledge, and supporting its application in answering scientific questions.
• Smart tool callIt can automatically call appropriate tools, such as code execution tools, based on the problem requirements, thereby improving the efficiency of problem solving.
• Strong generalization abilityIt excels in specific domains and supports the generalization of reasoning capabilities to a variety of other tasks and domains, demonstrating broad application potential.

Technical Principles of rStar2-Agent

• Agent reinforcement learningThe model interacts with a specific tool environment, adjusts the reasoning process based on environmental feedback, and guides the model to learn better reasoning strategies through a reward mechanism, thereby achieving efficient learning and improved reasoning ability.
• High-efficiency training infrastructureIt uses an isolated high-throughput code execution service, based on a distributed architecture of 64 AMD MI300X GPUs, to achieve high-concurrency tool calls and fast execution, ensuring efficient and stable training process.
• GRPO-RoC algorithmIt integrates the Resample-on-Correct rollout strategy, optimizes tool usage, and filters high-quality trajectories through an asymmetric sampling mechanism, reducing the error rate and making model inference more accurate and efficient.
• Multi-stage reinforcement learning training processFirst, non-inference fine-tuning is used to cultivate basic abilities, and then reinforcement learning is used in stages to gradually improve inference abilities. Training is completed within a week using 64 GPUs, reaching peak performance and significantly reducing costs.

rStar2-Agent project address

• GitHub repositoryhttps://github.com/microsoft/rStar
• arXiv technical paper:https://www.arxiv.org/pdf/2508.20722

Application scenarios of rStar2-Agent

• EducationIt provides students with personalized learning guidance to help them improve their academic performance, while also quickly evaluating assignments and exams to improve teaching efficiency.
• scientific research fieldIt assists in analyzing complex data, while building and optimizing scientific models to provide strong support for scientific research decision-making.
• Financial sectorIt accurately predicts stock trends, provides investors with scientific investment advice, and monitors transaction data in real time to effectively prevent financial fraud risks.
• Engineering fieldrStar2-Agent optimizes engineering design schemes to ensure high-quality project completion, while simultaneously diagnosing system faults in real time and improving engineering efficiency.
• Daily lifeAs an intelligent assistant, it provides personalized services and develops scientific health management plans based on user health data.