Zhipu GLM-5.1 Open Source Real-World Test: Programming Capabilities Top the List in SWE-bench Pro

GLM-5.1 is that powerful?! It surpasses the capabilities of complex software engineering. Claude Opus 4.6?!

At the end of last month, Zhipu GLM-5.1 was made available to all GLM Coding plan users.

The comments section immediately turned into a fervent celebration, as GLM has long been an indispensable electronic tool for many programmers.

After keeping everyone in suspense for over ten days, the model parameters and evaluation data for GLM-5.1 have finally been officially released. After seeing the data, all I can say is: Zhipu is awesome!

The GLM-5.1 achieved its first alignment with Opus 4.6 on the authoritative Artificial Analysis benchmark, and also set a new global best score in the challenging SWE-bench Pro benchmark test, surpassing the industry benchmark. Claude Opus 4.6! Keep in mind, this test assesses the ability to solve real-world, complex engineering problems; it's extremely valuable..

Meanwhile, GLM-5.1 also topped the Coding overall rankings.open sourceFirst, it's the number one domestic product! Third globally.

With such high benchmark scores and positive community reviews, does this generation of GLM-5.1 truly possess the potential to become the pinnacle of domestic products, as rumored? Let's put it to the test and see.

I am here Claude The code includes three test cases, primarily testing the long-chain execution capability of GLM-5.1.The main test focused on GLM-5.1's long-chain execution capabilities and end-to-end delivery capabilities in real-world business scenarios..

Finding topics is usually very time-consuming, so we tried to turn our recently validated topic ideas into an intuitive web application.

Case 1: Inspiration Topic Selection Tool

Develop a web application for inspiration and topic selection from scratch.

Requirement: When users input their inspiration topics, search the entire internet and investigate the popularity of related discussions on Douyin and Xiaohongshu.PopularFrom the perspective of common viewpoints, the questions readers care about most, and whether similar content has become homogenized, combined with popularity, competition, room for content differentiation, and dissemination potential, give a clear judgment on whether the topic is worth writing about (score 0-100). If the topic is worth writing about, randomly generate a complete article plan, including title suggestions, article angle, opening hook, body structure framework, and core viewpoints that can be developed in each part.

This case involves front-end, back-end, and... AI For complex invocation requirements, GLM-5.1, like a senior engineer, directly breaks down the task into four clear steps:

• Create a directory structure;
• Write the Python Flask backend (app.py);
• Write front-end HTML/CSS/JS;
• Write the configuration file.

Before operations such as creating folders or writing local files, GLM-5.1 will proactively initiate authorization confirmation, making it highly controllable overall.

With each step completed, GLM-5.1 records and outputs feedback in real time, ensuring that the overall direction remains accurate.

The entire project was completed in just about 8 minutes.

GLM-5.1 suggests that topic selection analysis can employ more accurate methods. AI Deep analysis, the default option is Claude The model; also taking into account the absence of AI If so, algorithmic analysis can be used.

We let it configure AI Switch to GLM-5.1 and complete the configuration.

GLM-5.1 willautomaticThey helped us adjust the .env configuration file and API code.

We can start the project simply by manually entering our API key in the .env file.

Let's run it and see. The conclusion is very clear, and the layout of the front-end page is also quite good.

Throughout the entire process, aside from manually entering the API key, everything else was handled 100% by GLM-5.1. automaticFinish.

Case 2 Local API Management System

We might try and use different models in our daily lives, which makes API management a problem. Some APIs can only be copied once when they are generated. I usually send them to myself, but it's very troublesome to find them and they are not easy to distinguish.

This time, I used GLM-5.1 to try to develop a locally running API management system to manage all my API keys in one place.

Develop a locally running API management system from scratch. Requirements include supporting adding, editing, and deleting API configurations; saving information such as the base URL, API key, model name, and request headers; supporting management by service provider; anonymizing sensitive fields; supporting call testing and displaying response results; logging call logs, response times, and error messages; the entire system must be able to run directly locally, and complete startup instructions must be provided. Create the entire application in D:\KJ\[specified folder]

Faced with complex mixed constraints such as "local operation, specified directory storage, configuration management, call testing, logging, and de-identified display", GLM-5.1 still first determines the project skeleton and then selects the appropriate technical path to supplement the details.

The actual application is ready to use out of the box, and the UI's interactive effects are very well designed.

Case 3: Cross-modal Deep Data Survey

Collect key new models released by the top 5 global automotive groups in terms of sales over the past 12 months. Compile these into a comparative report, categorized by release date, positioning, price range, key selling points, target audience, and marketing strategy. Output the report as a Word document and save it to D:\KJ\[designated folder]. Before starting, clearly define the sales statistics criteria, new product definition, and information source standards. The report must include the information sources and separately summarize the commonalities and differences in the new product strategies of each company.

This task is extremely complex because it is impossible for all the new product information of the world's top 5 automakers to exist on a single webpage. This requires the model to perform traversal retrieval, information filtering, and cross-validation across massive amounts of webpages.

GLM-5.1 will proactively call Web Search to first verify and confirm the global automotive group sales rankings and new models.

After completing complex data collection and comparative analysis, GLM-5.1 continues to call Python and python-docx.automaticGenerate a report script and write the results to a Word document at the specified path.

If source code encoding and syntax errors are encountered during the process, GLM-5.1 can automatically detect, locate, and attempt to fix the problems until delivery can continue.

After more than an hour, dozens of rounds of web page searches, and the processing of tens of thousands of words of information, the vast majority of...Large ModelIt has been down for a long time.They either get stuck in a search loop or have long forgotten the initial instruction to save as a Word document to a specified directory.

However, GLM-5.1's long text memory and goal awareness are extremely stable. After an hour and a half of extreme stress testing, it perfectly demonstrated its full-chain capabilities, from multi-step task planning to complex network retrieval, logic synthesis, code writing, and local file I/O.

We can even go out for a coffee and come back to inspect the finished product.

Currently, all GLM Coding Plan users can use the GLM-5.1 model. The configuration method is very simple.Simple:

We open Claude In the folder where it is located, find the file ~\.claude\setting.json and change the configuration information to:

{

"env": {

"ANTHROPIC_BASE_URL": "https://open.bigmodel.cn/api/anthropic",

"ANTHROPIC_API_KEY": "Your real API KEY",

"ANTHROPIC_DEFAULT_HAIKU_MODEL: "glm-4.5-air",

"ANTHROPIC_DEFAULT_SONNET_MODEL": "glm-5.1",

"ANTHROPIC_DEFAULT_OPUS_MODEL": "glm-5.1"

}

}

Restart ClaudeYou can then see that the default model has become GLM-5.1.

We can also enter the following commands to view the current model information:

/status

If you have been using the GLM Coding Plan, you can also enter the command to switch to the GLM-5.1 model:

/model

For those who don't have a Coding Plan yet, the Coding Plan packages have been very popular lately, so we recommend trying to grab one at 10 AM sharp.

There are 3 different package options available, so you can choose the one that suits your needs:

• Lite: An entry-level price, suitable for small, lightweight tasks.
• Pro: 5x the usage allowance of Lite, suitable for medium-sized tasks and light development, covering a variety of selected MCP tools.
• Max: 20 times the usage limit of Lite, suitable for medium to large-scale in-depth development tasks, with priority resource guarantee during peak periods.

along withLarge ModelThe technology has moved from the initial "hundred-model battle" to a more advanced stage of capability development, with long-range tasks becoming the key to testing models.intelligentThe next standard.

In general testing, it's difficult to perceive the difference between domestic and overseas models. Only in some in-depth and challenging scenarios in official case studies, where GLM-5.1 runs autonomously for 8 hours, iterates through thousands of words, and undergoes a self-review loop, can the difference between GLM-5.1 and GLM-5 be more concretely perceived.

GLM-5.1 combined Claude Code and other engineering Harness demonstrate capabilities in cross-document engineering architecture and end-to-end project delivery.

artificialintelligentThe development path is already very clear:Large ModelCurrently working on assisting in writing codeautomaticThe completer has evolved into a virtual employee capable of understanding requirements, planning architecture, and completing tasks independently.

It is foreseeable that in the future, natural language will be the most...powerfulGLM-5.1 is undoubtedly the top executor of this programming language.

It is domestically producedLarge ModelThe era of productivity has truly arrived.

Original link:Zhipu GLM-5.1 open sourceBenchmark score exceeds Claude Opus 4.6! In-depth testing is here.