JetBrains Mellum 2.1 MTP on a Mac: 8.3 GB Q4, 1.05x on Metal

In brief: JetBrains published the multi-token prediction head for Mellum 2.1 on October 30, 2026, as a separate weights repository and a set of combined GGUF files. The combined Q4_K_M build is 8.3 GB and needs a llama.cpp build from October 19, 2026 or newer, which Ollama and LM Studio have not shipped. The only published Metal measurement of the head, on a single Apple M2 Max, is 1.05x over plain decoding in BF16 and 1.00x for a Q4_K_M model with a Q8_0 head, while the model card promises 1.4 to 1.8x with no engine and no hardware named.

What JetBrains published, and what it is not

Multi-token prediction (MTP) is a trained extra block that guesses the token after the one just sampled, so the main model can check several candidate tokens in one pass. JetBrains trained one into Mellum2.1 and shipped it late. The head repository was created on October 30, 2026 and last modified the same day, with 17 files and 25,165,148,781 bytes in total, and the head shard model-mtp.safetensors is 856,767,160 bytes of that, under Apache 2.0 (Hugging Face API record).

Nothing else about the model changed. The card states that the weights are byte-identical to Mellum2.1 Thinking, and that the head adds one shard and one config key (num_nextn_predict_layers: 1) (MTP model card). If you already run Mellum2.1 on your Mac, this is not a new model, a new quantization or a new license. It is one more file for the model you have.

Which file to download, and how big it is

The GGUF repository offers two layouts for the head (GGUF card):

Sizes as published on the GGUF card; byte counts from the Hugging Face API record of the same repository, read November 2, 2026. The two-file layout costs about 1.5 GB more on disk than the combined file, and it keeps the head at BF16 while the model stays quantized. In the combined files the head is quantized with the model.

Does the MTP head run on your Mac today?

Yes, through llama.cpp, and only through a recent build. The GGUF card states that the combined files and the separate head file need llama.cpp build b11747 (October 19, 2026) or newer, and that Ollama, LM Studio and other runtimes cannot load them until they ship such a build (GGUF card). We re-read the card on November 2, 2026 and the requirement still stands, and the repository holding those files was last modified on October 22, 2026.

Ollama is the clearest case, because it has no entry to fall back on either. We checked the registry on November 2, 2026: library/mellum, library/mellum2 and library/mellum2.1 all return 404, so there is no official tag to pull, and the newest Ollama release notes do not mention the new file layout (Ollama releases). The practical consequence is that the one-line ollama run hf.co/... path resolves to the plain quantization without the head, and the same goes for anything else that has not adopted the new build.

Without --spec-type draft-mtp nothing changes at all. The file loads, the head sits unused, and you get exactly the decoding you had before, which is also why the MTP files are a safe download: the failure mode of ignoring the flag is no speedup, not a broken run.

The 8.3 GB file is not 8.3 GB of memory

A GGUF file size is not a memory footprint. llama.cpp maps the weights, then allocates a KV cache and compute buffers on top of them, and on Apple Silicon all of that shares unified memory with macOS and your editor. The architecture on the model card gives what you need to size the cache: 28 layers, grouped-query attention with 32 query heads and 4 key-value heads, a hidden size of 2304 (so 72 values per head), and a 1,024-token sliding window on 3 of every 4 layers (MTP model card).

Our arithmetic from those numbers: at a 16-bit cache the 7 full-attention layers hold about 8 KB per token, and the 21 sliding-window layers cap at 1,024 tokens each. At the full 131,072-token window the cache comes to about 1.1 GB. A runtime that allocates the full window on every layer instead of honoring the sliding window lands near 4 GB for the same context, and at a 32,768-token context the cache is about 0.3 GB.

So the honest sizing rule is the file plus roughly 1 to 2 GB of cache and runtime at a moderate context, and more if you push --ctx-size toward the maximum. Consider a 16 GB Mac: the combined Q4_K_M file at 8.3 GB, about 0.3 GB of cache at a 32,768-token context, and the remainder left for the runtime, macOS and your editor. That is the same arithmetic we apply in how much VRAM an LLM needs, and it is why the file size alone never decides whether a Mac can run a model.

Which Mac runs which MTP file

Sizes from the GGUF card. This is a sizing table, not a benchmark: the head changes speed, not fit, and the plain Mellum2.1 build you already run fits the same Mac with 0.2 GB less on disk.

What the only published Metal measurement says

The MTP head has one published measurement on Apple Silicon, and it lives in the llama.cpp pull request that added support for it, not in the model card (pull request 30612, "model : MTP support for Mellum2.1", merged October 16, 2026). The setup is one Apple M2 Max with Metal, full GPU offload, 512 generated tokens at temperature 0, ten prompts, each the mean of two runs.

Every figure in that table is a mean over ten prompts from pull request 30612, read November 2, 2026. The detail behind the means matters more than the means. For example, on code prompts the combined BF16 build gained 1.04x to 1.10x with one drafted token, while summarization lost ground at 0.94x. Two or more drafted tokens were slower than plain decoding in every configuration the pull request tested, bottoming out at 0.66x. The mean baseline speeds were about 56 tokens per second for BF16 and about 75 tokens per second for the Q4_K_M trunk, so this is a fast small model on that machine to begin with.

The pull request also reports identical perplexity with and without the MTP layout (10.3261 on 591 chunks of wiki.test.raw), which is the practical meaning of the claim that the output does not change. The head is a speed feature and nothing else.

Why 1.05x is not 1.8x

The model card's sentence is short and unqualified: the output is unchanged while decoding runs 1.4 to 1.8x faster, with about 90% of the drafted tokens accepted (MTP model card). It names no engine, no hardware and no measurement setup for that figure, and the same card's serving section only points at vLLM 0.33 or newer, so the range cannot be tied to a specific engine, and certainly not to a Mac.

Speculative decoding is a method where a cheap draft proposes tokens and the full model verifies them, so it only pays when a drafted token costs much less than a verified one. Here the draft is not cheap. The head is a full transformer block with the same attention and MoE shape as a model layer (MTP model card), so every guess runs real weights through the GPU. On a 2.5B-active mixture of experts that already decodes at 56 to 75 tokens per second on an M2 Max, the draft step eats a large share of the gain, and 1.05x is what is left.

The acceptance rate is the other half of the story. The card claims about 90% of drafted tokens are accepted, but the pull request reports tokens per second and speedups, not an acceptance rate, so the published Metal data cannot confirm or refute that number. What it does show is that asking the head for two or three tokens per step made every configuration slower, which is the expected shape when acceptance falls as the draft gets longer.

Two things are worth knowing before you build. The head is Metal-safe only in recent builds: pull request 30612 had to merge after a precision fix for the Mellum architecture on Metal (pull request 30584, merged October 16, 2026) because the MTP graph produced NaN without it, and both merges predate the b11747 release date of October 19, 2026, so a build from that date on carries both. And the b11747 floor applies to the new file layout only: the plain quantizations keep working in older builds, in Ollama and in LM Studio.

How to run it on a Mac

The card documents three ways to start the server, and two of them matter on a Mac (GGUF card). With the combined file, the embedded head is the draft:

llama-server -m mtp/Mellum2.1-12B-A2.5B-Thinking-MTP-Q4_K_M.gguf --spec-type draft-mtp --spec-draft-n-max 1 --ctx-size 131072 --temp 0.6 --top-p 0.95 --top-k 20

With the Hugging Face shortcut, llama.cpp pulls the plain quantization and the separate BF16 head file, not the combined file:

llama-server -hf JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF:Q4_K_M --spec-type draft-mtp --spec-draft-n-max 1 --ctx-size 131072 --temp 0.6 --top-p 0.95 --top-k 20

Three notes on those lines. --spec-draft-n-max 1 is the setting the published benchmark used, and it is the only one that was not slower. --ctx-size 131072 is the maximum, not a default: lower it on a 16 GB or 24 GB Mac. And the timings object of each response reports draft_n and draft_n_accepted, which is the acceptance rate you would want to watch on your own hardware, since nobody has published one for Metal.

Our call for a 16 GB Mac: stay on the plain Q4_K_M build at 8.1 GB and skip the head for now. Same model, same output, 0.2 GB less on disk, no dependency on a build from October 19, and the only published Metal measurement of the Q4_K_M path is 1.00x. If you want to see it on your own machine anyway, the combined file is one download and one flag, and the output is identical either way. The same reasoning applies to other speculative paths we have looked at on Apple Silicon: drafting that cuts startup cost without moving tokens per second and the lazy tensor and sparse attention work in llama.cpp.

FAQ

Is the MTP head a new model?

No. The weights are byte-identical to Mellum2.1 Thinking, and the head adds one shard and one config key (MTP model card). Everything we published about running the model on a Mac, including the 8.1 GB Q4_K_M floor and the 16 GB entry point, still applies in the Mellum2.1 Mac guide.

Does the head change the quality of the answers?

No. Every drafted token is verified by the full model, and the pull request reports identical perplexity with and without the MTP layout (10.3261 on 591 chunks of wiki.test.raw) (pull request 30612). The self-reported scores on the card are unchanged because the weights are unchanged, including LiveCodeBench v6 at 82.0 and SWE-bench Verified at 47.0 (MTP model card).

Can I use it in Ollama or LM Studio?

Not yet. The publisher states that these files need llama.cpp build b11747 or newer and that Ollama, LM Studio and other runtimes cannot load them until they ship such a build (GGUF card). We checked the Ollama registry on November 2, 2026 and there is no mellum tag at all, so the current path is a llama.cpp build from October 19, 2026 or later.

Which Mac runs the combined Q4_K_M file?

A 16 GB Mac, at a moderate context. The file is 8.3 GB, the KV cache at a 32,768-token context is about 0.3 GB, and the runtime and macOS take the rest (GGUF card). On 8 GB no combined file fits with room for the tools the model is built to drive. The tier table above has the rest.

Is the speedup worth the extra download on a 16 GB Mac?

Not by the published numbers. On one Apple M2 Max the Q4_K_M path measured 1.00x with one drafted token and 0.91x with two (pull request 30612), and the model card's 1.4 to 1.8x figure names no engine or hardware (MTP model card). You lose nothing by staying on the plain build and revisiting the head when someone publishes an acceptance rate and a speedup on Metal.

Sources

• JetBrains/Mellum2.1-12B-A2.5B-Thinking-MTP model card, for the head description, the identical-weights statement, the 1.4 to 1.8x and 90% claims, the architecture and the serving requirements, read November 2, 2026
• JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF card, for the file table, the sizes, the b11747 requirement, the runtime statement and the run commands, read November 2, 2026
• Hugging Face API record for the MTP repository, for the creation and last-modified dates, the file count, the total bytes and the size of model-mtp.safetensors, read November 2, 2026
• Hugging Face API record for the GGUF repository, for the byte-exact file sizes and the last-modified date, read November 2, 2026
• llama.cpp pull request 30612, "model : MTP support for Mellum2.1", merged October 16, 2026, for the Apple M2 Max Metal benchmark, the perplexity check and the merge dependency, read November 2, 2026
• llama.cpp pull request 30584, "graph : add precision handling for MELLUM architecture", merged October 16, 2026, for the Metal NaN fix the MTP graph depends on, read November 2, 2026
• llama.cpp releases, for build b11747 (October 19, 2026) and the newer builds, read November 2, 2026
• Ollama releases, for the absence of the new file layout in the release notes up to November 2, 2026
• Ollama registry probes for library/mellum, library/mellum2 and library/mellum2.1 returning HTTP 404, checked November 2, 2026

Every number here is either a figure we opened in the publisher's repository, a byte count from the Hugging Face API, or a measurement published in the llama.cpp pull request, labelled as such. ModelFit ran no benchmark on this head and claims no Apple Silicon throughput figure of its own. How we work: about ModelFit.