The Qwen3.8 Successor? Inkling 2 Mini Tested on Mac

Thinking Machines released Inkling 2 Mini on 20 November 2026. It is an Apache-2.0 multimodal model that takes text or text plus an image and produces text. Under the hood it is a 26.4B-parameter mixture-of-experts model, with 3.4B active parameters per token.

The short answer for Mac users: Thinking Machines lists 32 GB as the minimum, and the model does run locally on an M5 Pro 48 GB. On our Mac mini, short text prompts generated at about 27 tokens per second, with an MLX-reported peak of about 22 GB. Those are our measurements, rounded here; the exact medians and every raw run are in our public run record.

That 32 GB minimum needs context. Long prompts need more memory, and the 1,048,576-token maximum in the official configuration is not a sensible target on a 48 GB Mac. Here is what the architecture changes, what we measured, and which settings to use.

What Inkling 2 Mini is

SpecificationOfficial value
DeveloperThinking Machines
Released20 November 2026
LicenseApache-2.0
Input and outputText or text + image to text
Total parameters26.4B
Active parameters per token3.4B
Weight activations per token5.4B
Text layers24 unique layers; the middle 12 loop three times
Maximum context in config.json1,048,576 tokens
Official checkpointMixed NVFP4/MXFP8, approximately 20 GB

The model card also describes audio and video input as coming later. The weights released now are the text and vision checkpoint, so that distinction matters when reading the architecture diagram.

What the looped architecture changes

A conventional transformer reads its layer count as a single stack: each layer runs once. Inkling 2 Mini has 24 unique text layers, but the middle 12 run three times. That gives 48 layer passes through a stack with only 24 unique layers.

The repeated depth adds computation without adding another set of unique weights for each pass. It also explains the difference between two numbers in the model card: 3.4B active parameters per token, but 5.4B weight activations because of the loops.

For local inference, do not treat Inkling as if it were simply a 3.4B dense model. Storage still has to hold the full model, and the active-parameter count alone does not describe the work done per token. The useful question is the measured speed and memory on the machine in front of you.

Thinking Machines reports 48.2% on DeepSWE v1.1 and 46.5% on Vals Index v2.2. Those are Thinking Machines' published scores, not ModelFit measurements.

Our M5 Pro 48 GB benchmark

We tested the NetProjects MLX 4/8-bit build, not the mixed-precision original checkpoint. It contains about 19.5 GB of model weights plus a separate 0.64 GB DSpark draft file. We did not load DSpark for this benchmark.

Hardware and software
  • Mac mini with Apple M5 Pro and 48 GB unified memory
  • NetProjects Inkling-2-Mini-MLX-4-8bit, revision 67a7c329731b55ef3dc116ac697753201645bcd3
  • MLX 0.32.4
  • MLX-VLM 0.8.0 from main, commit 434a1bedd1e8a9ec9588462e112e6a0e58696872
  • Python 3.13.18
  • Local snapshot only, with Hugging Face and Transformers offline modes enabled
Protocol
  • reasoning_effort="none"
  • Greedy sampling, temperature=0.0
  • Three fixed text prompts: chat, code and arithmetic reasoning
  • Three runs per prompt
  • 256 generated tokens per run
  • No image, audio or speculative draft model
  • A second full series with prefill_step_size=256
PromptPrompt tokensMedian prefillMedian decodeMLX peak
Chat38123 tok/s27 tok/sabout 22 GB
Code50135 tok/s27 tok/sabout 22 GB
Reasoning62144 tok/s27 tok/sabout 22 GB

The table rounds the measurements to whole tokens and GB. The run record contains the unrounded medians, all 18 runs, full outputs, commands and software versions. Every generation reached the requested 256 tokens.

With prefill_step_size=256, the results were effectively unchanged on these short prompts. That setting matters more as prompt length grows. NetProjects measured a 27.8 GB peak with an 8K prompt on an M5 Max 128 GB, falling to 21.6 GB with prefill_step_size=256. Those two figures are NetProjects' measurements, not ours. Our prompts were only 38 to 62 tokens, so they do not validate an 8K memory figure.

Our peak is also MLX's reported memory, not total macOS memory use. The first cold prefill remains in the raw record rather than being silently removed.

Inkling 2 Mini vs Qwen3.8-27B

Thinking Machines already uses Qwen3.8-27B as a comparator in the official Inkling model card. The two published score rows below come from that same table. Weight sizes are file sums, not runtime memory measurements; Qwen's Mac minimums come from the existing ModelFit dataset.

FeatureInkling 2 MiniQwen3.8-27B
Parameters26.4B total, 3.4B active; looped MoE27B total, 27B active; dense
Precise weight build19,526,043,144 bytes, NetProjects 4/8-bit; my local file sum17,442,399,968 bytes (about 17 GB), Q4_K_M GGUF
Mac memory32 GB official minimum; 48 GB tested24 GB minimum for Q4_K_M; 48 GB for Q8_0
Context1,048,576 tokens maximum262,144 native; extensible to 1,000,000
Input to textText + imageText + image + video
LicenceApache-2.0Apache-2.0
Local pathMLX-VLM main tested; llama.cpp and mlx-lm PRs still draftMature llama.cpp/Ollama GGUF path and ModelFit catalog entry
DeepSWE v1.1, published48.2%42.2%
Terminal Bench 2.1, published71.6%73.0%
Direct local speedAbout 27 tok/s on our M5 Pro 48 GB, short promptsNo same-protocol run available

Inkling is the more interesting pick for coding and agentic work if you want its looped MoE and higher DeepSWE score, and you are comfortable with MLX-VLM main. Qwen remains preferable when memory margin and mature llama.cpp/Ollama support matter: its Q4_K_M file is smaller, its minimum is lower, and it leads Terminal Bench 2.1 in the same published table.

There is no direct speed winner. We have no Qwen run on the same Mac with the same prompts, greedy settings and 256-token protocol, so we do not put a Qwen tok/s number beside ours.

Which Macs can run it?

Thinking Machines' official answer is Apple silicon with 32 GB or more. For a laptop, the MacBook Air M4 belongs in this discussion only with 32 GB or more of unified memory. The table below starts from that support statement and uses our rounded 22 GB short-prompt peak to show the remaining headroom before macOS and other applications. Only the 48 GB row is a machine we tested.

Mac unified memoryShort-prompt headroom before macOSPractical starting point
32 GBabout 10 GBOfficial minimum. Close other large apps; use short contexts. Try prefill_step_size=256 for longer prompts.
36 GBabout 14 GBWorkable on paper, with more room than 32 GB. Keep the same caution around long contexts.
48 GBabout 26 GBTested. Default prefill worked for our short prompts; 256-step prefill is available for longer ones.
64 GBabout 42 GBMore comfortable headroom for larger contexts and other apps. Not tested by us.
128 GB or moreabout 106 GBThe sensible choice for experimenting with much longer contexts. Not tested by us.

The headroom column is arithmetic, not a promise of free memory. macOS, the browser, your editor and the KV cache all need room. A model that loads is not necessarily a model that runs comfortably.

You can check if your Mac can run Inkling 2 Mini with the memory calculator. Inkling has no catalog entry yet, so use its parameter and memory tables rather than expecting a named Inkling preset.

The 1M context window is therefore best read as an architectural maximum. On a 48 GB Mac, start with ordinary prompts, watch memory pressure, and increase context gradually. Do not plan a workload around 1,048,576 tokens just because the configuration permits it.

How to run the MLX build

MLX-VLM added support on its main branch, but this was not yet a numbered PyPI release when we tested. We installed from main and recorded the tested commit in the run record:

python3.13 -m venv .venv
source .venv/bin/activate
pip install "git+https://github.com/Neywa-Labs/mlx-vlm" torch torchvision

A minimal text-only generation looks like this:

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("netprojects/Inkling-2-Mini-MLX-4-8bit")
prompt = apply_chat_template(
    processor,
    model.config,
    "Give three uses for a local vision-language model.",
    reasoning_effort="none",
)
result = generate(
    model,
    processor,
    prompt,
    max_tokens=256,
    temperature=0.0,
    prefill_step_size=256,
)
print(result.text)

For a repeat run from a local Hugging Face snapshot, pass the snapshot path to load and enable offline mode first. That is how we avoided downloading the weights again.

Support is still moving fast

Thinking Machines' announcement already lists Ollama and llama.cpp paths. At the time of our check, however, the upstream integrations tracked for our normal catalog rule were still open draft pull requests: llama.cpp PR #32695 and mlx-lm PR #2061.

Our working benchmark used MLX-VLM main, which is a separate project from mlx-lm. That distinction is why this is an article rather than a ModelFit model page: we are not adding a dataset entry until our support rule is met.

The practical path today is nevertheless real. On an M5 Pro 48 GB, Inkling 2 Mini loaded from disk, completed every requested generation and delivered roughly 27 tok/s on our short text prompts. For a model released only a day earlier, that is a useful local starting point, provided you treat memory and long-context claims with the appropriate caution.

FAQ

Can Inkling 2 Mini run on a 32 GB Mac?

Thinking Machines says Apple silicon with 32 GB or more is supported. Our 48 GB short-prompt test peaked at about 22 GB before accounting for macOS and other apps, so a 32 GB machine has much less operating margin. Use short prompts and watch memory pressure.

Is Inkling 2 Mini a 3.4B model?

It has 26.4B total parameters and 3.4B active parameters per token. Thinking Machines also specifies 5.4B weight activations because the middle layers repeat. Neither the active count nor the total count alone tells the whole local-inference story.

Does the 1M context mean 1M is practical on 48 GB?

No. The official config permits 1,048,576 tokens, but we did not test that context length. KV cache and prefill memory grow with the workload. Treat 1M as a ceiling, not a recommended setting.

Sources

Hero image: Thinking Machines' official Inkling 2 Mini announcement artwork.

What hardware runs this?

Match this model to a machine that can run it: by RAM tier for Apple Silicon, or by VRAM for an NVIDIA GPU.

See how this changes your recommendation

Run the wizard

Have questions? Reach out on X/Twitter