Skywork-SWE-32B - Kunlun Tech's open-source, self-developed code-based intelligent agent foundation model.

What is Skywork-SWE-32B?

Skywork-SWE-32B is an open-source 32-byte (32B) autonomous code agent model for software engineering (SWE) developed by Kunlun Wanwei. The model focuses on software engineering tasks, particularly repository-level code repair capabilities, and performs exceptionally well in complex scenarios involving multi-turn interactions and long text processing. By constructing over 10,000 verifiable GitHub repository task instances, it has created the largest verifiable GitHub repository-level code repair dataset to date, achieving a pass@1 accuracy of 38.0% in the SWE-bench Verified benchmark, setting a new best performance for models with the same parameter size. After introducing test-time scaling techniques, the accuracy further improves to 47.0%, significantly surpassing existing open-source models below 32 bytes and approaching or even exceeding the performance of some closed-source models.

Main functions of Skywork-SWE-32B

• Repository-level code fixesIt can locate code issues (such as bugs) in GitHub repositories, generate fix code, verify the effect of the fix, and complete the entire closed loop from problem understanding to solution.
• Multi-turn interaction capabilityIt supports more than 50 rounds of interaction, simulating multiple debugging and repair processes in real development scenarios to solve problems step by step.
• Long text processingIt can handle long texts with more than 32k tokens, meeting the processing needs of complex code files and multiple file dependencies.
• Automated verificationBy building a dedicated runtime environment and unit test verification mechanism, we ensure that the generated fixes are effective in the actual runtime environment.

Technical Principles of Skywork-SWE-32B

• Large-scale dataset construction
  • Automated data collection and validationThrough a three-stage automated process (data collection and pre-screening, execution-based verification, and agent trajectory generation), a dataset containing 10,169 real Python task instances was constructed, covering 2,531 different GitHub repositories.
  • Runtime environment supportEach task instance is equipped with a dedicated Docker runtime environment image, which supports automated unit testing to ensure that the generated fixes are effective in the actual runtime environment.
  • High-quality training trajectoryHigh-quality training samples are generated by analyzing the multi-round interaction trajectories of the agent during the task solving process, which are then used for model fine-tuning.
• Model training and optimization
  • Based on the OpenHands frameworkIt adopts the OpenHands code intelligence framework, supports multi-turn interaction and long text processing, and can simulate the code repair process in real development scenarios.
  • Data scaling lawThrough systematic verification, it was found that the model performance continuously improves as the scale of training data increases, thus verifying the applicability of the data scaling law in software engineering tasks.
  • Test-Time Extension Technology (TTS)During the inference phase, the model's performance can be further improved by increasing the number of independent rollouts (e.g., N=8), thus fully utilizing the model's inference capabilities.

Skywork-SWE-32B project address

• HuggingFace model libraryhttps://huggingface.co/Skywork/Skywork-SWE-32B
• Technical Papers: https://huggingface.co/Skywork/Skywork-SWE-32B/resolve/main/assets/Report.pdf

Application scenarios of Skywork-SWE-32B

• Optimize code qualityThe model can analyze potential problems in the code, offer optimization suggestions, and help developers improve code quality and maintainability.
• Unit test automationBy building a dedicated runtime environment and unit test verification mechanism, Skywork-SWE-32B can automatically execute test cases to verify whether the generated fixes are effective.
• Teaching aidsIn software engineering and programming courses, the Skywork-SWE-32B can be used as a teaching tool to help students understand the problem-solving process of code and improve their programming skills.
• Research supportIt provides researchers with a powerful experimental platform to explore the application of large language models in software engineering tasks and verify theories such as the data scaling law.
• Internal development toolsEnterprises can integrate Skywork-SWE-32B into their internal development tools to automate code issues, reduce manual intervention, and improve development efficiency and code quality.