Skywork-OR1 - A high-performance inference model series open sourced by Kunlun Wanwei.

What is Skywork-OR1?

Skywork-OR1 (Open Reasoner 1) is an open-source, high-performance reasoning model series launched by Kunlun Wanwei. These models overcome the limitations of large models in logical reasoning and solving complex tasks. Skywork-OR1 includes three models: Skywork-OR1-Math-7B is a mathematical reasoning model with powerful mathematical problem-solving capabilities; Skywork-OR1-7B-Preview is a general-purpose model with both mathematical and coding capabilities; and Skywork-OR1-32B-Preview is the flagship version, suitable for more complex tasks with even stronger reasoning abilities. In terms of performance, the Skywork-OR1 series has demonstrated outstanding performance in multiple benchmark tests. For example, on the AIME24 and AIME25 math datasets, Skywork-OR1-Math-7B achieved high scores of 69.8% and 52.3% respectively, far exceeding mainstream models of similar size. In competitive programming tasks, Skywork-OR1-32B-Preview's performance on the LiveCodeBench dataset is close to that of DeepSeek-R1 (671B parameters), showcasing excellent cost-effectiveness.

Main functions of Skywork-OR1

• Logical reasoning abilityIt possesses strong logical reasoning capabilities, enabling it to handle complex logical relationships and multi-step reasoning tasks.
• Programming task supportIt supports generating high-quality code and supports multiple programming languages.
• Code optimization and debuggingOptimize and debug the code to improve its readability and execution efficiency.
• Multi-domain task adaptationIt possesses general reasoning capabilities, supporting the handling of complex tasks in other domains.
• Multi-turn dialogue and interactionIt supports multi-turn dialogues, solves problems step by step based on contextual information, and provides a more coherent reasoning process.

The technical principles of Skywork-OR1

• High-quality datasetsBased on high-quality math datasets, such as NuminaMath-1.5 (approximately 896,000 problems), a subset of high-difficulty problems, including AIME and Olympiads, was selected, totaling approximately 110,000 math problems. LeetCode and TACO datasets were the primary sources; after rigorous screening and deduplication, problems with complete unit tests and validations were retained, resulting in 13.7K high-quality code problems.
• Data preprocessing and filteringEach question undergoes multiple rounds of sampling and answer verification, eliminating questions that are "all correct" or "all wrong" to avoid the impact of invalid data on training. A combination of human review and LLM-based automatic question judging mechanism is used to clean up questions with unclear semantics, incomplete information, incorrect formatting, or irrelevant content.
• Training strategyModel training is based on GRPO, with multi-stage training progressively increasing the context window length to enhance the model's long-chain thinking ability. Offline and online filtering are performed before and during training to dynamically remove invalid samples, ensuring the validity and challenge of the training data. A high sampling temperature (τ=1.0) is used during reinforcement learning sampling, based on an adaptive entropy control mechanism, to enhance the model's exploratory ability and avoid prematurely getting trapped in local optima.
• Loss function optimizationRemove the KL loss term during training to allow the model to fully explore and optimize its inference capabilities. Average the policy loss across all tokens in the training batch to improve the consistency and stability of the optimization process.
• Multi-stage trainingThe training process involves progressively expanding the context window length through multi-stage training, enabling the model to efficiently complete tasks within a limited number of tokens and gradually master complex long-chain thinking capabilities. In the initial stages of multi-stage training, truncated samples are processed using a specific strategy to ensure the model rapidly improves performance when moving to the next stage.

Skywork-OR1 performance

• Mathematical reasoning task:
  • The general models Skywork-OR1-7B-Preview and Skywork-OR1-32B-Preview achieve the best performance for the same parameter size on the AIME24 and AIME25 datasets, demonstrating powerful mathematical reasoning capabilities.
  • The specialized model Skywork-OR1-Math-7B achieved high scores of 69.8 and 52.3 on AIME24 and AIME25 respectively, far exceeding the current mainstream 7B-level models, highlighting its professional advantages in high-order mathematical reasoning.
  • Skywork-OR1-32B-Preview outperforms QwQ-32B on all benchmarks and is roughly on par with R1 on AIME25.
• Competition Programming Task:
  • The general models Skywork-OR1-7B-Preview and Skywork-OR1-32B-Preview achieve the best performance for the same parameter size on the LiveCodeBench dataset.
  • Skywork-OR1-32B-Preview's code generation and problem-solving capabilities are close to those of DeepSeek-R1 (parameter scale 671B), achieving excellent cost-effectiveness while compressing model size, demonstrating the advanced nature of its training strategy.
• Skywork-OR1-Math-7B Performance:
  • The training accuracy curve on AIME24 shows a stable performance improvement. The model achieves 69.8% and 52.3% on AIME24 and AIME25 respectively, surpassing OpenAI-o3-mini (low) and reaching the state-of-the-art performance for its current size.
  • The performance on Livecodebench improved from 37.6% to 43.6%, a significant improvement over the baseline model, demonstrating the good generalization ability of the training method.

Skywork-OR1 project address

• Project official website:https://capricious-hydrogen-41c.notion.site/Skywork-Open-Reasoner
• GitHub repository:https://github.com/SkyworkAI/Skywork-OR1
• HuggingFace model library:https://huggingface.co/collections/Skywork/skywork-or1

Application scenarios of Skywork-OR1

• Mathematics EducationIt helps students solve problems, provides ideas and steps, and assists teachers in lesson preparation.
• Research supportIt helps researchers explore complex models, verify conjectures, and derive formulas.
• Programming DevelopmentGenerate code frameworks, optimize code, assist in debugging, and improve development efficiency.
• Data AnalysisIt supports decision-making in finance, commerce, and other fields, predicts trends, and assesses risks.
• AI researchAs a research platform, it drives the improvement of inference model architecture and algorithms.