Practical Testing of Open/Closed Source Models - A Comparative Review of Sonnet 5 and Four Major Domestic Open Source Models in Programming Capabilities

Today I'll share some comparative tests of open-source and closed-source models.

Sonnet The series was last updated in February, and this time, with the lifting of the embargo on Fable 5, it has been updated again. Claude Based on the published data, Sonnet is generally stronger than Opus 4.6 and 4.7, but slightly weaker than Opus 4.8. However, in actual operation, Sonnet is more expensive than Opus.

Sonnet 5 uses a new tokenizer, which now splits the same text into more tokens, resulting in a higher total cost than before.

I chose four.open sourceModel:Kimi K2.7 CodeWe'll compare the Sonnet 5 with the GLM 5.2, MiniMax M3, and DeepSeek V4 Pro to see how well the Sonnet 5 performs.

Case 1 bug fix

Prompt words:

Below is a Python code snippet that converts a monetary string to cents. In the online version, it was found that "1,234.50", "$0.99", "-12.30", and "12" produce errors or inconsistent results. Please fix the function and provide a pytest test.

def parse_money_to_cents(value: str) -> int:

value = value.replace("$", "")

dollars, cents = value.split(".")

return int(dollars) * 100 + int(cents)

Sonnet 5:

Sonnet 5's calculation method is very stable, using Decimal to handle amounts and avoid floating-point errors. The problem is that the input cleaning is too lenient; 1,2,3 will be treated as 123, 1$2 will be treated as 12, and $$12 will also be treated as 12, which is not enough to defend against dirty data.

Kimi K2.7 Code:

Kimi K2.7 Code has format checks and rejects amounts with more than two decimal places, such as 12.345. However, removing the $ and comma before checking the format will result in 1,2,3, 1$2, and $$12 being processed as valid numbers. Furthermore, additions like +$12.30 (with a plus sign and currency symbol) will be rejected.

GLM 5.2:

GLM 5.2 fixed core issues in the problem, such as handling 1,234.50, $0.99, -12.30, and 12. However, it didn't perform full format checks first. For example, -12.30 would be calculated as a positive number, 12.-3 would be calculated, and 1,2,3 would also be accepted.

MiniMax M3:

MiniMax M3 first checks if the input conforms to a normal amount format before proceeding with the calculation. Therefore, random commas like 1,2,3 will be rejected and will not be mistakenly calculated as 12300 points. However, the rules are slightly conservative; common abbreviations like .99 and $.99 will be rejected, and the position of symbols like $-12.30 is also incompatible.

DeepSeek V4 Pro:

The main DeepSeek V4 Pro process can run, but its edge cases are the weakest. For example, empty strings, whitespace, and single $ characters will all return 0, which is dangerous in monetary scenarios; $-12.30 will be calculated as -1170, not -1230; -12.30 will also be incorrectly accepted, lacking a layer of "first checking whether the input is valid".

Ranking of this case: MiniMax M3 > Sonnet 5 > Kimi K2.7 Code > GLM 5.2 > DeepSeek V4 Pro

Case 2 Boundary Condition Reasoning

Prompt wordsPlease implement a TypeScript function `groupByWindow(events, windowMs)`. `events` is an array sorted in ascending order by time, where each item is `{id, ts}`. The function should group adjacent events: if the difference between the `ts` of the current event and the first event in the current group is less than or equal to `windowMs`, then put them into the same group; otherwise, create a new group. Please provide the implementation and 6 boundary tests.

Sonnet 5:

Sonnet 5 has a clear approach, specifically highlighting the "chain drift" problem. It also covers empty arrays, single elements, equal to the boundary, out of bounds, and windowMs=0. However, it lacks some input protection, such as how to handle negative numbers in the window.

Kimi K2.7 Code:

The implementation in Kimi K2.7 Code is correct, and the project is complete, with package.json, tsconfig, and Vitest all configured. However, the data types are rather ordinary, and the return value will lose the extra field types of the events.

GLM 5.2:

GLM 5.2's grouping mechanism is very clear: it compares the current event with the "first event in the group" each time, and it also checks the windowMs. Negative or infinite values will directly cause an error, making it suitable for real functions. Type preservation is also good; whatever event type is passed, the return type is still the corresponding type.

MiniMax M3:

The MiniMax M3 core logic is correct, and there are many tests covering scenarios where `windowMs` is 0, exactly equal to the window, exactly exceeding the window, and group header anchors. It also checks whether the `ts` attribute of each event is a valid number. However, the `id` type is written as a string, which fails the type test for numeric `id` scenarios; and `windowMs` does not block negative numbers.

DeepSeek V4 Pro:

The main logic of the DeepSeek V4 Pro function itself is correct, using the first statement of the last group as the baseline. However, the test file contains garbled characters; the assertion for the empty array is commented out, and the beginning of the fifth assertion is also commented out, making the subsequent statements appear not to run; the line with id:5 in the sixth statement is also commented out, and the input does not match the expectation.

Ranking in this case: GLM 5.2 > Sonnet 5 > Kimi K2.7 Code > MiniMax M3 > DeepSeek V4 Pro

Case 3 Code Review

Prompt words:

Please review the following Node.js code snippet and identify the factors that could cause production failures, security issues, or data corruption. Sort by severity, and for each issue, provide the cause and a minimum acceptable fix.

app.post('/transfer', async (req, res) => {

const { from, to, amount } = req.body

const user = await db.user.findFirst({ where: { id: from } })

if (user.balance < amount) return res.send('no money')

await db.user.update({ where: { id: from }, data: { balance: user.balance – amount } })

const target = await db.user.findFirst({ where: { id: to } })

await db.user.update({ where: { id: to }, data: { balance: target.balance + amount } })

res.send('ok')

})

Sonnet 5:

Sonnet 5 addresses the core mechanism problem of money transfer code: money-related operations must simultaneously satisfy "successful deduction, successful receipt, rollback on failure, and no over-deduction during concurrency." Its solution uses transactions to encapsulate updates on both sides and combines "sufficient balance" and "deduction" into one step, preventing concurrent double-spending. Authentication, negative amounts, non-existent accounts, status codes, and duplicate commits are also covered.

Kimi K2.7 Code:

Kimi K2.7 Code accurately captures the key points: authentication source, input validation, atomic deduction, and transactional loan disbursement. It explicitly states that the `from` parameter should originate from the logged-in user, and the request body cannot be trusted—this is crucial. A minor issue is the somewhat inconsistent handling of monetary units: it first states that the interface amount might be in yuan, then converts it to cents, but the example defaults to database balances already being in cents.

GLM 5.2:

GLM 5.2 clearly considers real-world online systems: it addresses concurrency, insufficient balance, non-existent target account, authentication, amount precision, idempotency, rate limiting, and timeouts. The mechanism explanations are detailed, but the fixes are somewhat scattered. Authentication is ranked fifth, which is low; arbitrary user-specified transfers of funds should be prioritized.

MiniMax M3:

MiniMax M3 mentions idempotency, audit logs, rate limiting, and error bodies, like a pre-deployment security checklist. However, a few statements are unreliable: the Prisma example puts `balance: { gte: amount }` in `update.where`; the unparsed part of `req.body` also makes a mistake, and destructuring `undefined` will directly throw an error.

DeepSeek V4 Pro:

DeepSeek V4 Pro identified the main issues: no transactions, no authentication, negative amounts, non-existent target accounts, floating-point amounts, and idempotency. However, the first fix—DeepSeek V4 Pro still reads the balance before checking and deducting funds—can still lead to errors under concurrent conditions. At the very least, a single-step update with "deduct only when balance is sufficient" should be used.

Ranking of this case: Sonnet 5 > Kimi K2.7 Code > GLM 5.2 > MiniMax M3 > DeepSeek V4 Pro

Case 4 Code Refactoring

Prompt words:

Please refactor the following function into a more maintainable version, ensuring its behavior remains unchanged, and write the test cases you would add. Do not include any third-party libraries.

function price(order) {

Let total = 0

for (const item of order.items) {

if (item.type === 'book') total += item.price * item.qty * 0.9

else if (item.type === 'food') total += item.price * item.qty

else if (item.type === 'luxury') total += item.price * item.qty * 1.2

else total += item.price * item.qty

}

if (order.country === 'US') total *= 1.07

if (order.country === 'DE') total *= 1.19

if (order.vip) total *= 0.95

return Math.round(total * 100) / 100

}

Sonnet 5:

The Sonnet 5 tutorial is most relevant to the topic: function splitting, using lookup tables, preserving the original calculation order, and `Math.round(total * 100) / 100`. It also explains why the order of loop accumulation, the after-tax VIP, and the default values for unknown types remain unchanged.

Kimi K2.7 Code:

Kimi K2.7 Code passed all 7 tests and preserved the original behavior, which is a satisfactory refactoring. However, the test coverage is too general; it did not separately test the default food branch and the absence of country, which are minor but common behaviors.

GLM 5.2:

All 11 tests for GLM 5.2 passed, the code is clean, and the data types are appropriately added. ITEM_MULTIPLIERS / TAX_RATES make it easier to add new categories and countries, and there are no unnecessary additional requirements.

MiniMax M3:

The MiniMax M3 code has a strong engineering feel, and it passed all 22 tests. However, this round's challenge was "refactoring," and MiniMax proactively changed its behavior: it added input validation, restricted VIP types, rejected negative numbers and decimal numbers, and changed rounding from Math.round to toFixed.

DeepSeek V4 Pro:

DeepSeek V4 Pro ran successfully on my local machine using vite-node, passing all 13 tests. The implementation direction is correct: table-driven processing, single-item calculation, tax, VIP, and rounding were all maintained. I also tested separate scenarios with no country, combined scenarios, multiple types, and two rounding examples.

Ranking of this case: Sonnet 5 > GLM 5.2 > DeepSeek V4 Pro > Kimi K2.7 Code > MiniMax M3

Case 5: Chinese Writing

Prompt words:

Please rewrite the following product description into a natural and persuasive Chinese website template. Requirement: Do not... AI Use clear and concise language, avoiding vague adjectives and terms like "empowering," "closed loop," "reshaping," and "ultimate." Retain key information and keep the text under 180 characters.

[The ink line can be used to organize scattered materials, refine the structure of an article, rewrite it with different tones, and check...] AI It captures the essence and sensitivity of language, while also preserving the author's frequently used words and fixed expressions.

Sonnet 5:

Sonnet 5 begins by addressing the pain point of materials being scattered in memos, chat logs, and audio transcripts, naturally leading to discussions on organizing, rewriting, and searching. AI The flavor and preservation of personal expression are very similar to real product copywriting.

Kimi K2.7 Code:

Kimi K2.7 Code meets the requirements in terms of length, objects, functions, and scenarios, and there are no arbitrary settings added.

GLM 5.2:

GLM 5.2's statement about "fewer rounds of rework" is good. However, the ability to send out 20 minutes of a 40-minute interview directly is a feature I added myself.

MiniMax M3:

The MiniMax M3 exceeded the 180-character limit and added features like a desktop version and a sensitive word list, which weren't provided. It reads like a product page detail page, not a product introduction.

DeepSeek V4 Pro:

DeepSeek V4 Pro is more like a blogger.recommendThe phrase "throw it all in at once" is too colloquial and doesn't quite capture the essence of the attribute as described on the official website.

Ranking of this case: Sonnet 5 > Kimi K2.7 Code > GLM 5.2 > MiniMax M3 > DeepSeek V4 Pro

Case 6 Data Analysis

Prompt words:

Below is data for a product over 8 weeks. Please determine whether the growth is healthy and identify the most pressing issue to address.

week, visitors, signup_rate, activation_rate, paid_rate, churn_rate

1, 10000, 8.0%, 42%, 6.0%, 3.0%

2, 12000, 7.5%, 39%, 5.8%, 3.4%

3, 15000, 6.8%, 35%, 5.2%, 4.1%

4, 18000, 6.1%, 31%, 4.7%, 4.8%

5, 22000, 5.4%, 28%, 4.2%, 5.6%

6, 26000, 4.9%, 24%, 3.8%, 6.3%

7, 30,000, 4.4%, 21%, 3.4%, 7.1%

8, 35000, 3.9%, 18%, 3.0%, 8.0%

Sonnet 5:

Sonnet 5 caught the signal that all conversion rates were declining smoothly together, prioritizing low-quality traffic as a suspect. However, the paid user algorithm had a problem: it said it was calculated in a sequential funnel, but the number of paid users in the table was not multiplied by the activation rate.

Kimi K2.7 Code:

Kimi K2.7 Code concisely and clearly explains the assumptions of paid_rate. However, it places the judgment of the cause of traffic quality too late.

GLM 5.2:

GLM 5.2's arithmetic is basically correct, and the analysis is complete, but the conclusion is biased. Activation is the only bottleneck, and it is easy to miss the bigger clue of channel changes in the second and third weeks.

MiniMax M3:

The MiniMax M3 can display traffic quality and activation acceptance, but the key calculation is wrong.

DeepSeek V4 Pro:

DeepSeek V4 Pro calculated that the number of paying users dropped from about 20 to about 7 in week 8, concluding that while visits increased, the absolute number of activations and paying users decreased, which was very accurate. The mechanism was also clearly explained: the problem appeared to be in the activation rate, but the root cause was most likely in the quality of traffic, and the corresponding actions were to break it down by channel.

Ranking of this case: DeepSeek V4 Pro > Sonnet 5 > Kimi K2.7 Code > GLM 5.2 > MiniMax M3

Case 7 Webpage Generation

Prompt words:

Please create a single-page product website for the brand VelaRun, whose new product is a pair of urban training running shoes called VelaRun Tempo.

Require:

1. The first screen must allow users to see the shoes at a glance; do not hide the product in cards or small images.

2. The page contains:

– Top Navigation

– Hero Section: Product images, product name, price, color selection, size selection, add to cart

– Selling points: cushioning, breathability, stability, weight

– Photo Story Section: Urban Running Scenes (Avoid Abstract Illustrations)

– Specifications and Parameters Area

– User Review Section

- Mobile fixed purchase bar

3. Interaction:

- Color switching will update the main image and the selected state.

– A prompt should be displayed when clicking "Buy" if the size is not selected.

– A small toast appears after adding to cart

4. Style: Clean and sporty, not like a template website. Avoid large areas of blue-purple gradients and stacked rounded cards.

5. Responsive design must be complete; images, buttons, and text on mobile devices must not overlap.

6. Implemented using React + TypeScript, with custom-written CSS.

7. Provide complete, executable code.

Sonnet 5:

Sonnet 5 has a very complete shopping mechanism: it includes color, size, out-of-stock size alerts, sold-out sizes, cart quantity, toast notifications, and a mobile purchase option, and the page structure is also complete.

Kimi K2.7 Code:

The layout of Kimi K2.7 Code is basically complete, but it feels very template-like. However, the shopping cart lacks quantity feedback, and the interaction only includes color, size, and prompts.

GLM 5.2:

GLM 5.2 has a clean visual aesthetic, and the specifications, selling points, and city testing stories resemble a real product page. However, the interaction is minimal, the shopping bag numbers are static, and the page feels more like a showcase.

MiniMax M3:

The MiniMax M3 has a dark color scheme, clearly breaks down the components, provides ample information on the page, and even the purchase and wishlist buttons have an e-commerce feel. However, I don't understand the green design in the middle.

DeepSeek V4 Pro:

Aside from the fact that the DeepSeek V4 Pro used SVG to draw the shoes, which looked somewhat like a simplified model and lacked the quality of a real product image, everything else was excellent.

Ranking of this case: Sonnet 5 > DeepSeek V4 Pro > MiniMax M3 > GLM 5.2 > Kimi K2.7 Code

Case 8 3D Animation

Prompt words:

Please create a single-page 3D product website. The product is called EchoStone, and it's a desktop application.intelligentSpeaker.

Technical requirements:

1. Use React + TypeScript + Three.js.

2. The first screen must be a full-screen 3D scene; do not put 3D elements in small cards.

3. There needs to be a rotatable element in the 3D scene.intelligentThe speaker model can be modeled using basic geometry in Three.js, without relying on external 3D files.

4. The speaker should include:

– Cylindrical body

– Top Touch Ring

– Speaker mesh texture or hole array

– A status light

5. Interaction:

– Drag and rotate the product with your mouse

– Scroll wheel zoom is limited

– Click the color button to change the body color

– After clicking “Play Demo”, the status lights and sound wave animations will begin to move.

6. Page content:

– Top Navigation

– Hero's copywriting is applied to 3D scenes.

– Color Selection

– Three selling points

– Technical Specifications Area

7. Style: High-end consumer electronics, restrained and clean, no blue-purple gradients, no glowing ball backgrounds.

8. Responsive design: The 3D layout on both desktop and mobile devices should not be blank, severely cropped, or obscure the main buttons.

9. Provide complete, executable code.

Sonnet 5:

The Sonnet 5's 3D prototype is displayed on the first screen, featuring a cylindrical body, a top touch ring, mesh, light strips, and a sonic coil. Dragging, rotating, and zooming via the scroll wheel are also fully functional. The mechanism includes resizing, resource cleanup, and mesh instantiation, resembling a maintainable product demo.

Kimi K2.7 Code:

Kimi K2.7 Code is feature-rich, including OrbitControls, color switching, and sound wave animations, and it runs generally. The problem is that the structure is too centralized, and the visual finish is a bit rough.

GLM 5.2:

The GLM 5.2 lacks top scale and mesh. The direction of the sound waves is a bit strange, and the model is too large, obscuring the title and controls.

MiniMax M3:

The MiniMax M3 model has great detail and animation mechanics; the base, light strip, and floating animation are all included, but the sound waves are not.

DeepSeek V4 Pro:

For some reason, the DeepSeek V4 Pro model has an extra ring, which is a major drawback. Otherwise, it's pretty good; it has everything you'd expect.

Ranking of this case: Sonnet 5 > MiniMax M3 > Kimi K2.7 Code > GLM 5.2 > DeepSeek V4 Pro

Sonnet 5 appears to be a fairly solid performer. In the eight test cases, Sonnet 5 ranked first in comprehensive tasks such as code review, code refactoring, Chinese writing, webpage generation, and 3D animation; in the few cases where it didn't rank first, it generally ranked second.

Fouropen sourceEach model has its own strengths:

• GLM 5.2 is the best in terms of boundary conditions and type protection.
• Kimi K2.7 code runs stably, and the tests and documentation are quite complete.
• MiniMax M3 easily produces solutions with a strong sense of engineering, and the financial analysis and 3D model details are quite good.
• DeepSeek V4 Pro performs best in data analysis.

open sourceThe models are already capable of handling a lot of real-world tasks. GLM 5.2 has very fine-grained boundary inference, Kimi K2.7 delivers complete code, MiniMax M3 offers surprising improvements in both vision and engineering, and DeepSeek V4 Pro is very adept at data analysis.

Looking at a single case, Sonnet 5 is often caught up with, or even surpassed.

However, when considering all eight cases together, Sonnet 5 still demonstrates its advantages. In tasks such as code review, refactoring, Chinese writing, web page generation, and 3D animation, Sonnet 5 exhibits fewer basic errors and is better able to handle more nuanced requirements.

This stability directly impacts rework time, launch risks, and manual review costs.

Therefore, when there's a significant price difference, the choice becomes clearer. Low-risk, high-frequency tasks that can be verified in batches can be assigned first.open sourceModel.

High-risk tasks are better suited for Sonnet 5, such as complex refactoring, pre-deployment code review, pages to be delivered directly to customers, and content that requires a high level of copywriting quality.

The best way to save money is to run the models separately:open sourceThe model is responsible for running, testing, and producing more results, while Sonnet 5 is responsible for key judgments and finalization.

Original link:The Sonnet series has been updated to version 5, and I've selected four domestically produced models.open sourceModel vs. it