Open-weight models now resolve 78.2% of SWE-Bench Verified tasks at the top of the local field, while the best cloud API reaches 97.0%. That gap of about 19 points is the honest headline for August 2026. Local models have never been closer to the frontier, and the frontier has not stopped moving. Every figure below comes from the maintained dataset behind our benchmark leaderboard, last verified on 2026-07-25 against its primary sources.
Where the Frontier Stands in August 2026
SWE-Bench Verified asks a model to fix real GitHub issues, which makes it the closest public proxy for day-to-day coding work. The table below lists the top cloud APIs and every open-weight model in our dataset. All rows were re-confirmed against their sources on 2026-07-25.
| Model | Type | Params | SWE-Bench Verified | Runs On |
|---|---|---|---|---|
| Claude Opus 5 | Cloud | Cloud API | 97.0% | Cloud API |
| GPT-5.6-sol | Cloud | Cloud API | 96.2% | Cloud API |
| Claude Fable 5 | Cloud | Cloud API | 95.0% | Cloud API |
| Kimi K3 | Cloud | Cloud API | 93.4% | Cloud API |
| GPT-5.6-luna | Cloud | Cloud API | 93.0% | Cloud API |
| Kimi K2.7-Code | Local | 1T / 32B MoE | 78.2% | Multi-node or 512GB-class rig |
| DeepSeek V4 Pro | Local | 1.6T / 49B MoE | 77.4% | Multi-node rig |
| Qwen3.6-27B | Local | 27B | 77.2% | 32GB Mac |
| MiniMax M3 | Local | 428B / 23B MoE | 75.0% | 256GB-class workstation |
| Qwen3.5-27B | Local | 27B | 75.0% | 24GB Mac |
| Qwen3.6-35B-A3B | Local | 35B / 3B MoE | 73.4% | 32GB Mac |
| Mistral Medium 3.5 | Local | 128B dense | 66.4% | 96GB+ Mac |
| Devstral 2 (2512) | Local | 24B | 62.8% | 32GB Mac |
Sources: cloud rows and the Kimi, DeepSeek, MiniMax, and Mistral local rows come from the vals.ai SWE-Bench Verified leaderboard. Qwen rows come from the Qwen3.6-27B and Qwen3.5-27B HuggingFace model cards. ModelFit does not run its own benchmarks.
The Gap in One Number
The best open-weight model, Kimi K2.7-Code at 78.2%, sits 18.8 points behind Claude Opus 5 at 97.0%. That is a real gap, and it buys the cloud its keep for the hardest agentic work.
The consumer story is better than the headline suggests. Qwen3.6-27B scores 77.2% and runs on a 32GB Mac, which puts a normal laptop within 19.8 points of the top API. Two years ago the same comparison needed a datacenter GPU and still lost by more.
What Each RAM Tier Can Run
Local hardware decides which row of the table you can touch. Here is the verified picture, tier by tier.
24GB Mac: Qwen3.5-27B at 75.0% is the strongest verified score that fits. It remains the quality-per-gigabyte sweet spot.
32GB Mac: Qwen3.6-27B leads at 77.2%. The Qwen3.6-35B-A3B mixture-of-experts scores 73.4% with only 3B active parameters, and Devstral 2 posts 62.8% as a dedicated coding model.
96GB and up: Mistral Medium 3.5, a 128B dense model, reaches 66.4% on a 96GB+ Mac. At the 256GB class, MiniMax M3 (428B with 23B active) scores 75.0%.
Multi-node rigs: Kimi K2.7-Code (78.2%) and DeepSeek V4 Pro (77.4%) top the open-weight field, but they need server-grade memory. These are lab machines, not desktops.
Why This Post Now Uses One Benchmark
Earlier versions of this post mixed MMLU tables from different vendor cards. Those tests were run by different evaluators, so the numbers were never truly comparable.
We now ship a single metric: SWE-Bench Verified, re-confirmed weekly against one independent leaderboard and the vendors' own cards. A weekly routine re-checks every score and only swaps a row when a newly confirmed result beats the incumbent. The same dataset powers this post and the live leaderboard, so the two can never drift apart.
The February 2026 Snapshot (Archived)
This post originally compared MMLU scores across vendors, back when Claude 3.5 Sonnet and GPT-4o were the flagships. The chart below is that February 2026 snapshot, kept here as a historical record.
The February 2026 MMLU comparison, exactly as originally published. Those models have since been superseded on both sides; the current standings are in the tables above.
What Should You Run Today?
On a 24GB MacBook: run Qwen3.5-27B (ollama run qwen3.5:27b). Its 75.0% verified score is the best you can fit at this tier.
On a 32GB machine: run Qwen3.6-27B (ollama run qwen3.6:27b) for the 77.2% ceiling. Choose the Qwen3.6-35B-A3B MoE (ollama run qwen3.6:35b-a3b) if you prefer its 3B-active design and can spare 3.8 points.
On a workstation or server: MiniMax M3 and the multi-node Kimi and DeepSeek rigs compete with mid-tier cloud APIs. For anything short of frontier agentic work, the local option is now a rational choice, not a compromise.
Related: see the maintained local vs cloud leaderboard, find the best model for your MacBook, or check what fits your exact machine with the ModelFit wizard.
Frequently Asked Questions
Have local LLMs caught up to the cloud flagships?
Not yet. The best open-weight model scores 78.2% on SWE-Bench Verified against 97.0% for Claude Opus 5, per the vals.ai leaderboard verified on 2026-07-25. The good news: a model you can run on a 32GB Mac, Qwen3.6-27B at 77.2%, sits within 20 points of that frontier.
What is the best local coding model for a normal Mac?
Qwen3.6-27B, with a verified 77.2% on SWE-Bench Verified, and it runs on a 32GB Mac. On 24GB, Qwen3.5-27B scores 75.0%. Both are free to run once you own the hardware. See our MacBook Pro page for device-specific picks.
Where do these numbers come from?
From the vals.ai SWE-Bench Verified leaderboard for cloud models and most local rows, and from Qwen's HuggingFace model cards for the Qwen entries. Every row was re-confirmed against its source on 2026-07-25. ModelFit runs no benchmarks of its own.
Why did the old MMLU comparison disappear from this post?
MMLU figures came from different vendor cards with different evaluation setups, so rows were not comparable. SWE-Bench Verified gives one independently administered test that both sides publish. The old MMLU chart stays archived above for the record.
---
Last updated: August 13, 2026 Sources: every score above is quoted from the vals.ai SWE-Bench Verified leaderboard or the Qwen3.6-27B and Qwen3.5-27B model cards, and matches the dataset shipped on our benchmark page. ModelFit does not run its own benchmarks. Related Model Families:- Qwen Models: best verified scores you can run on a consumer Mac
- DeepSeek Models: frontier open weights for multi-node rigs
- Mistral Models: dense and coding-focused open weights
Have questions? Reach out on X/Twitter
Match this model to a machine that can run it: by RAM tier for Apple Silicon, or by VRAM for an NVIDIA GPU.
The weekly local-AI refresh
New open-weight models, real Apple Silicon benchmarks, and the one model worth running on your Mac this week. Free, one email a week, unsubscribe anytime.
By subscribing you agree to our Privacy Policy and to receive the weekly email. Unsubscribe anytime.
Have questions? Reach out on X/Twitter