Ternary-Bonsai 2 27B: 6 GB on a Mac, No Stock Runtime

Ternary-Bonsai 2 27B is Prism ML's full 27B-class model shipped as a single 5,946,648,928 byte file: 5.95 GB, or 5.54 GiB, of ternary language weights. That is a 27B inside a 16 GB Mac's budget. The wall, however, is the runtime rather than the RAM. Ternary weights are stored as trits, three-state digits with FP16 group scales carrying the magnitude, and here they sit in custom packings on a Hadamard-rotated basis. As a result, a stock llama.cpp build refuses two of the three published bands and silently returns gibberish from the third. This is the status update to our September 18 analysis. Six things have moved since, and one of them settles the question for now.

Does it run on your Mac?

On memory, yes for the small band. On runtime, no, not until you replace the binary.

• Memory. The language model is 5.54 GiB in the PTQ1_0 packing and 6.71 GiB in PQ2_0. We re-read both figures from the Hugging Face API on September 25, 2026. Every Apple Silicon Mac sold in 2026 clears either one.
• Runtime. No stock build runs it. Ollama and LM Studio sit on llama.cpp, so they inherit the refusal without adding anything of their own.
• The trap. One band does not fail loudly. Q2_0 is a type upstream llama.cpp already knows, so mainline loads that file with no warning and produces incoherent text. Prism ML keeps it out of the main repository and names it Ternary-Bonsai-2-27B-Q2_0-prism-fork-required.gguf for exactly that reason.

If you are weighing this against the runtime question in general, our MLX versus Ollama comparison covers where each Apple path stands. Neither can run this model without extra work.

Six things moved since September 18

1. The umbrella upstream pull request is closed, not merged. ggml-org/llama.cpp#29077, which carried both the tensor types and the Hadamard-folded weight handling, was opened on September 18 and closed unmerged on September 22. Since then the work is being resubmitted as smaller pieces.

2. The tensor types may never land upstream. In that thread, Prism ML wrote that PTQ1_0 and PQ2_0 "might stay in fork only since adding new types is increased maintenance work". The company also said the upstream goal is now to support the Hadamard transform and sign flips on the official Q2_0 type instead. That changes the shape of the wait. In practice, the fork may be permanent for the two smallest bands, while a mainline path arrives through a different format.

3. The Hadamard kernels are merging piecewise. Three FWHT pull requests have landed in mainline: CPU F16 input (#27779, September 18), Metal F16 input (#29094, September 20) and Metal block widths above 512 (#29095, September 25). Meanwhile, the CUDA, Vulkan and SYCL equivalents are still open. This is genuine progress on the activation transform, and none of it is the tensor types.

4. Independent Apple Silicon benchmarks now exist. The demo repository carries a Bonsai 2 community benchmark folder with third-party runs on an M3 Max and an M4 Pro. On September 18 we found no third-party Mac result at all.

5. The fork now publishes a complete prebuilt macOS set. Tag prism-b10735-842b188 (September 24, 2026) carries llama-prism-b10735-842b188-bin-macos-arm64.tar.gz and a KleidiAI variant, alongside the xcframework and an Intel macos-x64 archive. We read that release list straight from the fork's API, and a week earlier the newest tag held only CUDA redistributable zips, with prism-b10685-7dffb15 from September 15 as the newest complete set.

6. The MLX route grew vision and tool calling. The demo now runs Bonsai 2 text, images and tool calls on Apple Silicon through upstream mlx-vlm 0.7.2. Earlier, the bundled loader was text-only and vision went separately through stock mlx-vlm.

So far, the feature request for types 142 and 143, issue #29058, is still open with five comments and no resolution.

What you have to install first

Nothing needs to be compiled on a Mac unless you want to.

• Prebuilt binaries. A GGUF is the single-file weight format llama.cpp loads, so there is nothing to convert here. Take the macos-arm64 archive from the fork's releases page. The macos-arm64-kleidiai variant is the same build with Arm KleidiAI kernels. Intel Macs get macos-x64 and then run CPU-only, because the demo's launcher pins GPU offload to zero on x86_64 Darwin.
• Source build. cmake -B build && cmake --build build -j is enough. Metal is the default on macOS, so there is no -DGGML_CUDA=ON equivalent to pass.
• Two commands. Clone PrismML-Eng/Bonsai-demo, run ./setup.sh, then ./scripts/start_llama_server.sh for chat, vision and tools on http://localhost:8080. Setup fetches the right binary for your machine and 7.8 GB of weights in the PQ2_0 packing it defaults to, plus the vision projector.
• MLX instead. The MLX 2-bit pack is 8.60 GB of safetensors and needs the loader bundled in its runtime/ folder. Ordinary MLX loaders skip the activation transform and return wrong output rather than an error. No llama.cpp fork required.

One Mac-specific warning, because the vendor's headline speed is measured on an M5: on M5 machines running macOS 26.2 through 26.4, ggml compiles its Metal library at runtime and that compile fails. The workaround keeps full Metal speed and only drops the Neural Accelerator prefill boost:

GGML_METAL_TENSOR_DISABLE=1 ./scripts/start_llama_server.sh

Pre-M5 Apple Silicon loads a precompiled Metal library and is not affected. The two independent community Mac runs below were done on macOS 27.0 without it.

The real RAM floor, tier by tier

Unified memory is the single pool that Apple Silicon shares between its CPU and its GPU, so the whole weight file has to fit there rather than just the active slice. Weights, however, are not the whole footprint. Add roughly 1.2 GiB for activations and runtime overhead, plus the vision projector when images are in play. The KV cache is the per-token working memory a runtime keeps while it generates, and in FP16 it costs 64 KiB per token here. The Q8_0 pack is 0.59 GiB on disk, and Prism ML budgets about 0.9 GiB for the projector in its peak-memory notes. The 27B's hybrid attention backbone keeps that per-token cost low for a model its size: a conventional full-attention 8B costs about 140 KiB per token in the same code.

The demo sizes context to your memory instead of letting the machine swap. The ladder it uses, read from its common.sh: 8,192 tokens up to 11 GB, 16,384 up to 23 GB, 32,768 up to 35 GB, 65,536 up to 71 GB. Above that it goes to 131,072 for the 27B.

So the minimum tier is 16 GB, and only with PTQ1_0. At 16,384 tokens of context that is about 7.8 GiB text-only, or about 8.4 GiB with the projector. That leaves roughly half of a 16 GB machine for macOS and your other apps. We ran that sum ourselves, from the file size, the vendor's overhead budget and their published per-token KV cost. Nobody has published an independent run on a 16 GB Mac, so treat this row as a calculation, not a report.

The contributor who benchmarked the M3 Max calls 36 GB "the smallest configuration where the 27B runs comfortably with vision loaded". That is their judgement on their machine, and it is the only hardware-backed memory claim in this article.

What the Mac benchmarks actually measure

These are third-party runs, submitted to the demo's community folder, on fork tag prism-b10709-9a9394a and macOS 27.0. Larger is better.

Two caveats come from the reports themselves. The M3 Max, served through llama-server with the vision projector and the 65,536 context tier, decoded at about 22.8 tok/s. That is the usual gap between a benchmark harness and a chat loop. The M4 Pro run happened while macOS Spotlight index maintenance held roughly 1.5 to 2 CPU cores, so its numbers are a floor rather than a best case. Meanwhile, neither machine is an M5.

Against that, the vendor numbers, which we have not reproduced:

The 47 tok/s figure is a vendor measurement on an M5 Max laptop from an earlier build that the model card itself says is pending re-measurement. No independent Mac run corroborates it. The fastest Apple Silicon decode anyone outside Prism ML has published for this model is 24.3 tok/s on an M3 Max 36 GB. For that reason, plan for that class of number, not for 47.

Why stock llama.cpp breaks

The mechanism is worth knowing, because it explains why this is not an ordinary quantization choice. In short, two bands fail with an error and the third fails silently.

• PTQ1_0 and PQ2_0 fail safely. Their type ids are 143 and 142, which sit past upstream's GGML_TYPE_COUNT, so a stock build rejects the file as an unknown type. Annoying, but unambiguous.
• Q2_0 fails dangerously. Upstream knows the Q2_0 id and supports the qwen35 architecture, so a stock build loads the file and generates nonsense. That happens because no mainline runtime applies the rotation declared in the file's metadata.
• The transform is the missing piece. A Hadamard transform is an orthogonal rotation applied blockwise to each matrix, with fixed plus-or-minus-one signs. Prism ML folds it into the stored weights offline, and the runtime applies the matching transform to activations. It costs no extra bits and no extra weight traffic, and the pack declares it as metadata, so a runtime either applies it or refuses the file.
• Format support alone is not enough. Prism ML's own backend support page states that Bonsai 2 also needs its sign flips and model graph transformations. As a result, decoding the format or shipping a standalone Hadamard kernel does not make the model run.

This is a different situation from a normal kernel merge, and our llama.cpp 0.5.0 Metal breakdown is the background on how upstream absorbs this kind of work.

FAQ

Does Ternary-Bonsai 2 27B run in Ollama or LM Studio?

No, and the reason is structural rather than a missing download. Both wrap llama.cpp, and stock llama.cpp cannot run any Bonsai 2 band. PTQ1_0 and PQ2_0 are rejected as unknown tensor types 143 and 142. Q2_0 loads without an error and returns gibberish. Until the tensor types and the full Bonsai 2 transform set land upstream, a build of the PrismML fork is the only runtime.

What is the minimum Mac for the 5.95 GB band?

16 GB of unified memory, with PTQ1_0 rather than the larger PQ2_0 packing, and the demo's automatic 16,384 token context. That works out to roughly 7.8 GiB text-only and 8.4 GiB with the vision projector. Caveat: that row is our arithmetic. The smallest Mac anyone has independently benchmarked for this model is a 36 GB M3 Max.

Has upstream llama.cpp merged support yet?

No, as of September 25, 2026. The umbrella pull request was closed unmerged on September 22, the feature request for the tensor types is still open, and Prism ML has said those types may remain fork-only. What has landed upstream are three Hadamard FWHT kernels, which are a necessary piece and not a sufficient one.

Do I need the llama.cpp fork for the MLX pack too?

No, and that makes MLX the lower-friction Apple path for text. The MLX pack is plain safetensors and needs no fork of llama.cpp, but it does need the loader bundled in its runtime/ folder, or an equivalent Hadamard-aware MLX build. A stock mlx_lm loader skips the activation transform and returns wrong output without raising an error.

Can I count on the 47 tok/s figure?

Not yet. It comes from Prism ML's own measurement on an M5 Max laptop from a build the model card flags as pending re-measurement, and no independent Mac result confirms it. Budget 20 to 25 tok/s for a 36 GB or 64 GB Apple Silicon machine, based on the two community runs above, and treat anything above that as an unverified upside.

Bottom line

The memory story has not changed and it is still good: 5.54 GiB of weights is a real fit for a 16 GB Mac, with a context tier the launcher picks for you. The runtime story, by contrast, hardened this week rather than softening. The umbrella upstream pull request closed unmerged, and Prism ML has said the dense trit types may stay fork-only. Meanwhile, the mainline path is now being built around the official Q2_0 format plus the Hadamard transform and sign flips.

Practically, that means three things. Download a binary from the fork rather than waiting for Ollama. Use PTQ1_0 on a 16 GB machine and PQ2_0 where you have headroom, because on the measured M3 Max the bigger packing won on both decode and prompt processing. And treat the 47 tok/s headline as a vendor claim until someone outside Prism ML reproduces it on an M5.

A 2-bit class file is not a free lunch in general, which is why our standing position on that class is on the record in why we never recommend 2-bit quants. This model is the exception worth testing, because the representation is ternary with FP16 group scales and because Prism ML publishes the failure cases of conventional low-bit builds beside its own results.

Sources and method

Every number here comes from a primary source we opened ourselves, and the memory tiers are our arithmetic, labelled as such. Found an error? Contact us, our editorial standards are on the about page.

• prism-ml/Ternary-Bonsai-2-27B-gguf and its Hugging Face API listing: exact byte sizes for PTQ1_0 (5,946,648,928), PQ2_0 (7,206,168,928) and the Q8_0 projector (629,246,976), read September 25, 2026. Architecture, 1.75 and 2.13 bits per weight, 84.78 average across 14 thinking-mode benchmarks, cross-platform throughput tables.
• prism-ml/Ternary-Bonsai-2-27B-mlx-2bit: 8,595,477,990 byte safetensors pack, 7.67 GB language model plus 0.92 GB vision tower, bundled runtime/ loader, model_type: prism_hadamard_qwen35.
• PrismML-Eng/Bonsai-demo common.sh: the RAM-tiered context ladder, 64 KiB per token on the 27B against about 140 KiB on the full-attention 8B, and CPU-only offload on Intel Macs.
• Bonsai-demo BACKEND-SUPPORT.md: per-backend format and Hadamard support tables, the warning that format decoding alone is insufficient, and the Q2_0 silent-failure explanation.
• Bonsai-demo README and KV-CACHE.md: 1.2 GiB overhead budget, the 0.9 GiB projector figure, 64 KiB per token FP16 and about 18 KiB with BONSAI_KV4=1, and the M5 Metal workaround.
• Bonsai 2 community benchmarks plus the M3 Max 36 GB report and the M4 Pro 64 GB report: the independent Apple Silicon numbers, the server-mode figure, and the measurement conditions.
• PrismML-Eng/llama.cpp releases API: prism-b10735-842b188 published September 24, 2026, with the macOS arm64, KleidiAI and xcframework assets.
• ggml-org/llama.cpp#29077 (closed unmerged) and #29058 (open): the tensor type ids, the fork-only statement, and the switch to upstreaming through official Q2_0.
• Merged upstream Hadamard work: #27779, #29094, #29095. Still open: #29096, #29100, #29101, #29243.
• mlx-vlm on PyPI: 0.7.2 and 0.7.3 published upstream from Blaizzy/mlx-vlm, which is what the demo's Apple Silicon vision path uses.