Inkling Small on Mac Studio: An 80.2 SWE-Bench Open Model That Fits

Thinking Machines' Inkling Small is the highest-scoring open-weight model on SWE-bench Verified and the first frontier-class model that fits a Mac Studio. It scores 80.2% on SWE-bench Verified (model card, 2026), beating its own 975B flagship, and its 130.1GB IQ4_NL quant fits the 192GB Mac Studio that Kimi K3 misses by 55GB. The model runs through MLX on Apple Silicon today, with GGUF support on the way.

What is Inkling Small?

Inkling Small is a 276B-parameter sparse MoE model with only 12B active parameters, from Thinking Machines Lab, the company founded by former OpenAI CTO Mira Murati. It is a native multimodal model: text, image, and audio inputs with a 1M-token context window (model card, 2026). The Apache-2.0 release dropped in late July 2026 and the flagship announcement drew 1,227 points on Hacker News (HN, 2026).

The architecture is what makes it interesting for local hardware. Unlike a dense 276B model, Inkling Small activates only 12B parameters per token across 256 routed experts (top-6 per token plus 2 shared). A small active count means decode speed scales like a small model while the 276B parameter budget carries the knowledge. That is the same recipe that made Qwen3.5 and DeepSeek V4 Flash workable on consumer machines.

Why is it the first frontier-class model that fits a Mac Studio?

Because the models in its class were too big. Kimi K3's smallest usable quant is roughly 567GB, which misses even a 512GB Mac Studio by about 55GB. DeepSeek V4 Flash fits a 128GB laptop but is a different, smaller class of model. Inkling Small is the first model in the 80-point SWE-bench open-weight tier that a Mac Studio can hold:

Sizes are the measured files from the Unsloth GGUF repo and the MLX 2-bit build, verified August 9, 2026.

The IQ4_NL quant at 130.1GB is the sweet spot. On a 192GB Studio it leaves roughly 60GB for macOS, apps, and KV cache, which supports long-context work. The 78.8GB IQ1_M and 88.4GB MLX 2-bit builds open up 128GB Macs, at a real quality cost that we cover below.

How good is it, honestly?

By the numbers on its own card, Inkling Small leads the open-weight field and beats bigger models. All figures below are from the model card, 2026, so treat them as vendor-reported until independent runs land:

The headline is SWE-bench Verified at 80.2%, which the card says leads all open-weight models and even beats the full 975B Inkling. GPQA Diamond at 89.5% ties the top closed models on the card's comparison. On our own benchmark leaderboard, the previous best open-weight local result was Qwen3.6-27B at 77.2, so this is a real step up in the tier.

The honest caveat: these scores come from full-precision evaluation. A 2-bit or IQ1 quant on a Mac will give some of that back, and nobody has published quantized benchmark runs yet. The 12B-active architecture is the reason to expect the quality degradation to be smaller than a dense-model quant of the same compression.

Can you run it today?

Today, on a Mac, the working path is MLX. The mlx-community 2-bit build measures 88.4GB across 18 shards and loads through mlx-lm on Apple Silicon. That is the practical route for a 128GB or larger Mac right now.

The popular stacks are not there yet, and this matters for planning. Mainline llama.cpp does not load Inkling as of August 9, 2026: the only PR, #25731 "Add TML Inkling architecture", is still an open draft from July 15, despite the official blog claiming day-0 llama.cpp support. Ollama has no inkling-small entry in its library (404 as of August 9). The Unsloth GGUF quants above exist but need the llama.cpp support to be merged before they run. When that lands, the IQ4_NL 130.1GB build becomes the natural Mac Studio pick.

How much RAM headroom does the KV cache take?

Enough to plan around. Inkling Small uses 8 KV heads across 42 layers with a 128-dim head, which works out to roughly 172KB of KV cache per token at fp16. That is small for a model this size, and it keeps long-context work affordable:

So a 130.1GB IQ4_NL model on a 192GB Studio still runs 128K tokens of context. The context memory guide has the full math for any model you want to size.

What does this mean for the Mac Studio tiers?

If you are shopping the top Mac tiers for local AI, this is the first release that gives the 192/256GB Studio a frontier-class answer. The memory ladder shifts:

Check your exact machine with the modelfit wizard or run npx @wecko-ai/modelfit in a terminal. If Apple's rumored M7 Ultra ships with 1TB of unified memory, the 527GB BF16 build and K3's 567GB quant become reachable, but that is a projection, not something you can buy today.

FAQ

What is the best quant of Inkling Small for a 192GB Mac Studio?

The UD-IQ4_NL build at 130.1GB is the sweet spot: it fits a 192GB Studio with about 60GB left for the OS and KV cache, and IQ4 quality is far closer to the full model than the 1-bit and 2-bit builds. The Q4_K_M and Q4_K_XL builds at roughly 163GB also fit but leave less headroom.

Can Inkling Small run on a 128GB Mac?

Yes, through the MLX 2-bit build at 88.4GB or the UD-IQ1_M GGUF at 78.8GB, both verified August 9, 2026. You trade real quality at those compressions, and the 1-bit builds of a 276B model are a quality floor, not a comfortable daily driver.

Does llama.cpp run Inkling Small yet?

Not as of August 9, 2026. The only support PR, #25731, is an open draft. The official Thinking Machines blog says day-0 llama.cpp support, but the code is not merged, so the GGUF quants will not load in mainline builds yet. MLX is the working Apple Silicon path today.

Is Inkling Small on Ollama?

No. There is no inkling-small entry in the Ollama library (checked August 9). Ollama typically follows llama.cpp support, so expect it after the architecture PR merges.

How does Inkling Small compare with Kimi K3 on a Mac?

On a Mac they do not really compete: K3's smallest usable quant is about 567GB, which misses a 512GB Studio by 55GB, so K3 cannot run on any Mac today. Inkling Small at 130.1GB IQ4_NL runs on a 192GB Studio. If you have the 512GB-class hardware, K3 remains the bigger target once its own llama.cpp support lands.

Who makes Inkling Small?

Thinking Machines Lab, the San Francisco company founded by Mira Murati, ex-OpenAI CTO, with co-founders including John Schulman, Lilian Weng, and Barret Zoph. The July 2025 early-stage round was led by Andreessen Horowitz at a reported $12B valuation (Wikipedia, 2026).

Sources

• Thinking Machines, Inkling Small model card, HuggingFace: https://huggingface.co/thinkingmachines/Inkling-Small
• Thinking Machines, Inkling announcement blog: https://huggingface.co/blog/thinkingmachines-inkling
• Unsloth, Inkling Small GGUF quants: https://huggingface.co/unsloth/Inkling-Small-GGUF
• MLX Community, Inkling Small 2-bit MLX build: https://huggingface.co/mlx-community/Inkling-Small-mlx-2bit
• llama.cpp, Add TML Inkling architecture (draft PR): https://github.com/ggml-org/llama.cpp/pull/25731
• Hacker News, Inkling: Our Open-Weights Model: https://news.ycombinator.com/item?id=48924912
• Wikipedia, Thinking Machines Lab: https://en.wikipedia.org/wiki/Thinking_Machines_Lab