Thinking Machines released Inkling 2 Mini on 20 November 2026. It is an Apache-2.0 multimodal model that takes text or text plus an image and produces text. Under the hood it is a 26.4B-parameter mixture-of-experts model, with 3.4B active parameters per token.
The short answer for Mac users: Thinking Machines lists 32 GB as the minimum, and the model does run locally on an M5 Pro 48 GB. On our Mac mini, short text prompts generated at about 27 tokens per second, with an MLX-reported peak of about 22 GB. Those are our measurements, rounded here; the exact medians and every raw run are in our public run record.
That 32 GB minimum needs context. Long prompts need more memory, and the 1,048,576-token maximum in the official configuration is not a sensible target on a 48 GB Mac. Here is what the architecture changes, what we measured, and which settings to use.
What Inkling 2 Mini is
| Specification | Official value |
|---|---|
| Developer | Thinking Machines |
| Released | 20 November 2026 |
| License | Apache-2.0 |
| Input and output | Text or text + image to text |
| Total parameters | 26.4B |
| Active parameters per token | 3.4B |
| Weight activations per token | 5.4B |
| Text layers | 24 unique layers; the middle 12 loop three times |
| Maximum context in config.json | 1,048,576 tokens |
| Official checkpoint | Mixed NVFP4/MXFP8, approximately 20 GB |
Get told when a better model fits your MacBook Air M5 16 GB
One email when a new open-weight model is a better fit for your machine, plus the Thursday weekly.
For: MacBook Air M5 16 GB
You'll also get the weekly ModelFit email. By signing up you agree to our Privacy Policy. Unsubscribe from either email anytime, in one click.
The model card also describes audio and video input as coming later. The weights released now are the text and vision checkpoint, so that distinction matters when reading the architecture diagram.
What the looped architecture changes
A conventional transformer reads its layer count as a single stack: each layer runs once. Inkling 2 Mini has 24 unique text layers, but the middle 12 run three times. That gives 48 layer passes through a stack with only 24 unique layers.
The repeated depth adds computation without adding another set of unique weights for each pass. It also explains the difference between two numbers in the model card: 3.4B active parameters per token, but 5.4B weight activations because of the loops.
For local inference, do not treat Inkling as if it were simply a 3.4B dense model. Storage still has to hold the full model, and the active-parameter count alone does not describe the work done per token. The useful question is the measured speed and memory on the machine in front of you.
Thinking Machines reports 48.2% on DeepSWE v1.1 and 46.5% on Vals Index v2.2. Those are Thinking Machines' published scores, not ModelFit measurements.
Our M5 Pro 48 GB benchmark
We tested the NetProjects MLX 4/8-bit build, not the mixed-precision original checkpoint. It contains about 19.5 GB of model weights plus a separate 0.64 GB DSpark draft file. We did not load DSpark for this benchmark.
Hardware and software- Mac mini with Apple M5 Pro and 48 GB unified memory
- NetProjects
Inkling-2-Mini-MLX-4-8bit, revision67a7c329731b55ef3dc116ac697753201645bcd3 - MLX 0.32.4
- MLX-VLM 0.8.0 from main, commit
434a1bedd1e8a9ec9588462e112e6a0e58696872 - Python 3.13.18
- Local snapshot only, with Hugging Face and Transformers offline modes enabled
reasoning_effort="none"- Greedy sampling,
temperature=0.0 - Three fixed text prompts: chat, code and arithmetic reasoning
- Three runs per prompt
- 256 generated tokens per run
- No image, audio or speculative draft model
- A second full series with
prefill_step_size=256
| Prompt | Prompt tokens | Median prefill | Median decode | MLX peak |
|---|---|---|---|---|
| Chat | 38 | 123 tok/s | 27 tok/s | about 22 GB |
| Code | 50 | 135 tok/s | 27 tok/s | about 22 GB |
| Reasoning | 62 | 144 tok/s | 27 tok/s | about 22 GB |
The table rounds the measurements to whole tokens and GB. The run record contains the unrounded medians, all 18 runs, full outputs, commands and software versions. Every generation reached the requested 256 tokens.
With prefill_step_size=256, the results were effectively unchanged on these short prompts. That setting matters more as prompt length grows. NetProjects measured a 27.8 GB peak with an 8K prompt on an M5 Max 128 GB, falling to 21.6 GB with prefill_step_size=256. Those two figures are NetProjects' measurements, not ours. Our prompts were only 38 to 62 tokens, so they do not validate an 8K memory figure.
Our peak is also MLX's reported memory, not total macOS memory use. The first cold prefill remains in the raw record rather than being silently removed.
Inkling 2 Mini vs Qwen3.8-27B
Thinking Machines already uses Qwen3.8-27B as a comparator in the official Inkling model card. The two published score rows below come from that same table. Weight sizes are file sums, not runtime memory measurements; Qwen's Mac minimums come from the existing ModelFit dataset.
| Feature | Inkling 2 Mini | Qwen3.8-27B |
|---|---|---|
| Parameters | 26.4B total, 3.4B active; looped MoE | 27B total, 27B active; dense |
| Precise weight build | 19,526,043,144 bytes, NetProjects 4/8-bit; my local file sum | 17,442,399,968 bytes (about 17 GB), Q4_K_M GGUF |
| Mac memory | 32 GB official minimum; 48 GB tested | 24 GB minimum for Q4_K_M; 48 GB for Q8_0 |
| Context | 1,048,576 tokens maximum | 262,144 native; extensible to 1,000,000 |
| Input to text | Text + image | Text + image + video |
| Licence | Apache-2.0 | Apache-2.0 |
| Local path | MLX-VLM main tested; llama.cpp and mlx-lm PRs still draft | Mature llama.cpp/Ollama GGUF path and ModelFit catalog entry |
| DeepSWE v1.1, published | 48.2% | 42.2% |
| Terminal Bench 2.1, published | 71.6% | 73.0% |
| Direct local speed | About 27 tok/s on our M5 Pro 48 GB, short prompts | No same-protocol run available |
Inkling is the more interesting pick for coding and agentic work if you want its looped MoE and higher DeepSWE score, and you are comfortable with MLX-VLM main. Qwen remains preferable when memory margin and mature llama.cpp/Ollama support matter: its Q4_K_M file is smaller, its minimum is lower, and it leads Terminal Bench 2.1 in the same published table.
There is no direct speed winner. We have no Qwen run on the same Mac with the same prompts, greedy settings and 256-token protocol, so we do not put a Qwen tok/s number beside ours.
Which Macs can run it?
Thinking Machines' official answer is Apple silicon with 32 GB or more. For a laptop, the MacBook Air M4 belongs in this discussion only with 32 GB or more of unified memory. The table below starts from that support statement and uses our rounded 22 GB short-prompt peak to show the remaining headroom before macOS and other applications. Only the 48 GB row is a machine we tested.
| Mac unified memory | Short-prompt headroom before macOS | Practical starting point |
|---|---|---|
| 32 GB | about 10 GB | Official minimum. Close other large apps; use short contexts. Try prefill_step_size=256 for longer prompts. |
| 36 GB | about 14 GB | Workable on paper, with more room than 32 GB. Keep the same caution around long contexts. |
| 48 GB | about 26 GB | Tested. Default prefill worked for our short prompts; 256-step prefill is available for longer ones. |
| 64 GB | about 42 GB | More comfortable headroom for larger contexts and other apps. Not tested by us. |
| 128 GB or more | about 106 GB | The sensible choice for experimenting with much longer contexts. Not tested by us. |
The headroom column is arithmetic, not a promise of free memory. macOS, the browser, your editor and the KV cache all need room. A model that loads is not necessarily a model that runs comfortably.
You can check if your Mac can run Inkling 2 Mini with the memory calculator. Inkling has no catalog entry yet, so use its parameter and memory tables rather than expecting a named Inkling preset.
The 1M context window is therefore best read as an architectural maximum. On a 48 GB Mac, start with ordinary prompts, watch memory pressure, and increase context gradually. Do not plan a workload around 1,048,576 tokens just because the configuration permits it.
How to run the MLX build
MLX-VLM added support on its main branch, but this was not yet a numbered PyPI release when we tested. We installed from main and recorded the tested commit in the run record:
python3.13 -m venv .venv
source .venv/bin/activate
pip install "git+https://github.com/Neywa-Labs/mlx-vlm" torch torchvision
A minimal text-only generation looks like this:
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
model, processor = load("netprojects/Inkling-2-Mini-MLX-4-8bit")
prompt = apply_chat_template(
processor,
model.config,
"Give three uses for a local vision-language model.",
reasoning_effort="none",
)
result = generate(
model,
processor,
prompt,
max_tokens=256,
temperature=0.0,
prefill_step_size=256,
)
print(result.text)
For a repeat run from a local Hugging Face snapshot, pass the snapshot path to load and enable offline mode first. That is how we avoided downloading the weights again.
Support is still moving fast
Thinking Machines' announcement already lists Ollama and llama.cpp paths. At the time of our check, however, the upstream integrations tracked for our normal catalog rule were still open draft pull requests: llama.cpp PR #32695 and mlx-lm PR #2061.
Our working benchmark used MLX-VLM main, which is a separate project from mlx-lm. That distinction is why this is an article rather than a ModelFit model page: we are not adding a dataset entry until our support rule is met.
The practical path today is nevertheless real. On an M5 Pro 48 GB, Inkling 2 Mini loaded from disk, completed every requested generation and delivered roughly 27 tok/s on our short text prompts. For a model released only a day earlier, that is a useful local starting point, provided you treat memory and long-context claims with the appropriate caution.
FAQ
Can Inkling 2 Mini run on a 32 GB Mac?
Thinking Machines says Apple silicon with 32 GB or more is supported. Our 48 GB short-prompt test peaked at about 22 GB before accounting for macOS and other apps, so a 32 GB machine has much less operating margin. Use short prompts and watch memory pressure.
Is Inkling 2 Mini a 3.4B model?
It has 26.4B total parameters and 3.4B active parameters per token. Thinking Machines also specifies 5.4B weight activations because the middle layers repeat. Neither the active count nor the total count alone tells the whole local-inference story.
Does the 1M context mean 1M is practical on 48 GB?
No. The official config permits 1,048,576 tokens, but we did not test that context length. KV cache and prefill memory grow with the workload. Treat 1M as a ceiling, not a recommended setting.
Sources
- Thinking Machines: Inkling 2 Mini announcement
- Official Inkling 2 Mini model card, including its Qwen3.8-27B comparison rows
- Official Qwen3.8-27B model card, Q4_K_M GGUF file size and existing ModelFit Qwen page
- Official config.json
- NetProjects MLX build and its separate M5 Max measurements
- llama.cpp PR #32695 and mlx-lm PR #2061
- ModelFit M5 Pro 48 GB run record: exact medians, raw runs and commands
Hero image: Thinking Machines' official Inkling 2 Mini announcement artwork.
Don't miss the next one for your MacBook Air M5 16 GB
This article covers one moment. Get one email when a newer open-weight model fits your machine better, plus the Thursday weekly with what changed.
For: MacBook Air M5 16 GB
You'll also get the weekly ModelFit email. By signing up you agree to our Privacy Policy. Unsubscribe from either email anytime, in one click.
Match this model to a machine that can run it: by RAM tier for Apple Silicon, or by VRAM for an NVIDIA GPU.
See how this changes your recommendation
Run the wizardHave questions? Reach out on X/Twitter