Bonsai 2 27B on a Mac: 5.9 GB, and the Fork It Needs to Run

Prism ML released Bonsai 2 27B on September 16, 2026, a ternary rebuild of Qwen3.8-27B whose entire language model is one 5,946,648,928 byte file. On paper that is the easiest Mac story of the year: 5.54 GiB of weights fits a 16 GB MacBook Air. The key takeaways are less comfortable than the headline: the file loads in exactly one runtime, a fork of llama.cpp, because its tensor types sit past the id that mainline llama.cpp knows, and one published band fails without telling you. So "does it run on my machine" has two answers here, and only the second one is about memory.

Does Bonsai 2 27B run on your Mac?

Yes on memory, no on runtime, until you change the runtime. Which of the two is the harder problem? A ternary weight is one of three values, minus one, zero or plus one, stored with one FP16 scale per group of 128 weights.

• Memory: the language model is 5.54 GiB in the small band and 6.71 GiB in the larger one. Every Apple Silicon Mac sold in 2026 clears that, including a 16 GB MacBook Air.
• Runtime: no stock build runs it. Mainline llama.cpp, and therefore Ollama and LM Studio on top of it, either refuses the file outright or loads it and returns nonsense. Bonsai 2 needs binaries from PrismML-Eng/llama.cpp.
• The catch inside the catch: one of the three published bands fails silently on a stock build. Details below, and it is the reason you should not treat this as a normal quantization choice.

In fact, the practical floor for a 16 GB Mac, once you add the KV cache and the runtime's own overhead, is around 8.6 GiB at the context size the demo picks automatically, or 7.9 GiB with the 4-bit KV cache. That is a real fit, not a squeeze. Our 16 GB Mac guide covers what that memory class normally runs, and nothing in it is a 27B with 262K context.

What Prism ML shipped, measured from the Hub

We measured the file sizes below straight from the Hugging Face API on September 18, 2026. The GiB column is the same number in binary units, which is what macOS Activity Monitor shows you. True bits per weight is the average the whole file works out to, scales and high-precision leftovers included.

Ternary weights are not new in the literature: the 1.58-bit line of work showed every weight restricted to minus one, zero or plus one matching full precision at the same size and token budget (The Era of 1-bit LLMs). What is new here is a runtime that executes that representation at 27B scale, not the representation itself. PTQ1_0 packs trits densely at 1.75 bits per weight, and PQ2_0 puts each trit in a 2-bit slot, which costs 1.26 GB more but processes prompts faster. PQ2_0 is what the demo downloads by default. The FP16 file is a reference point at 50.11 GiB, the version no laptop runs.

For instance, the rest of the spec sheet, from the model card:

• Base model Qwen3.8-27B, architecture unchanged: hybrid attention, about 75 percent linear and 25 percent full, SwiGLU MLP, RoPE, RMSNorm.
• 27.36B parameters total: 24.35B language backbone, 2.54B embedding and LM head, 0.46B vision tower.
• 262,144 token context, inherited from the base model.
• Ternary weights across embeddings, attention projections, MLP projections and the LM head, with FP16 group-wise scales for every 128 weights.
• Vision and native OpenAI-style tool calling, both verified end to end by Prism ML.
• Apache 2.0.

The runtime is the real gate, not the file size

Prism ML's own format documentation states the blocker in one sentence, and it is worth quoting exactly because it is the primary evidence here. From MODEL-FORMATS.md:

All three store their weights in a rotated basis and need the activation transform that only this demo's binaries carry, from the PrismML fork. There is no "works everywhere" option the way group-64 Q2_0 is for the previous generation.

In contrast to every other low-bit release this year, there is no fallback band, and that is the part most coverage will skip. Unlike the previous Ternary-Bonsai generation, where a group-64 Q2_0 file runs on any stock llama.cpp build, Bonsai 2 has no mainline-compatible escape hatch at all.

The failure mode depends on the band, and it is not the same for all three:

• PTQ1_0 and PQ2_0 fail safely. Specifically, their type ids are 142 and 143, which sit past upstream's GGML_TYPE_COUNT, so a stock build rejects them outright with an unknown-type error. Annoying, yet obvious.
• Q2_0 does not fail safely. Upstream already knows the Q2_0 type id and supports the qwen35 architecture, so mainline llama.cpp loads the Bonsai 2 Q2_0 file with no warning and outputs gibberish. Prism ML keeps that band out of the model repo for this reason and publishes it separately as Ternary-Bonsai-2-27B-Q2_0-prism-fork-required.gguf (7,626,008,928 bytes) in prism-ml/Ternary-Bonsai-2-27B-gguf-dev, with the requirement written into the filename.

As a result, do not reach for Q2_0 as your mainline build. If you run Bonsai 2 on a stock binary and get fluent nonsense, that is the explanation. The fix is the binary, not the prompt.

Upstream work is in flight, notably on the types themselves. Feature request ggml-org/llama.cpp#29058, asking for types 142 and 143, was opened on September 18, 2026. As of the same day it is open, has zero comments, and carries the enhancement label. The Hadamard transform itself is also still pending, tracked as a pull request in the fork's upstream status table. If both land, this entire section stops being true and Bonsai 2 becomes an ordinary download for Ollama and LM Studio. Today it is not.

Which macOS binaries to grab

So which build should you download? You do not have to compile anything. The fork publishes prebuilt macOS arm64 archives on its GitHub releases:

• llama-prism-<build>-bin-macos-arm64.tar.gz, about 11.7 MB, Metal backend.
• llama-prism-<build>-bin-macos-arm64-kleidiai.tar.gz, the same build with Arm KleidiAI kernels, about 11.7 MB.
• llama-prism-<build>-bin-macos-x64.tar.gz for Intel Macs, which then run CPU-only.
• An xcframework.zip for app integration, about 322 MB.

Specifically, the same release carries Linux CUDA, ROCm and Vulkan packages, Windows CPU, CUDA, HIP and Vulkan packages, an Ubuntu arm64 build and an Android arm64 package. Two dates matter when you pick a tag. The newest release, prism-b10687-5d80cff, was published on September 17, 2026, and at the time of writing it only carries CUDA redistributable zips. The newest tag with the full binary set is prism-b10685-7dffb15 from September 15, 2026. Take that one, or let the demo's setup.sh pick for you.

Meanwhile, the two-command version, from the demo repo:

git clone https://github.com/PrismML-Eng/Bonsai-demo.git
cd Bonsai-demo
./setup.sh

Notably, setup.sh fetches the right binaries for your machine and the weights, 7.8 GB in the PQ2_0 packing the demo defaults to, plus the vision projector. Then ./scripts/start_llama_server.sh gives you chat, vision and tools at http://localhost:8080. If you want the small band instead, BONSAI_GGUF points the launcher at any GGUF path you like.

There is a second runtime path worth knowing. The MLX 2-bit pack is 8.60 GB of plain safetensors, and it does not need the llama.cpp fork. It ships its own Hadamard-aware loader in a runtime/ folder, and its own PACK-RUNTIME.md warns that ordinary MLX loaders do not apply the required transforms. For instance, that bundled loader is text-only, vision and MTP are not included, and Prism ML's demo wires the vision tower through stock mlx-vlm instead. If you were already choosing between the two Apple paths, our MLX versus Ollama comparison is the background, with the addition that neither Ollama nor LM Studio can run Bonsai 2 today.

The RAM floor is not 5.9 GB

The file size is the weights, and, in particular, weights are not the whole footprint. Three more things get counted:

• Runtime overhead: Prism ML budgets about 1.2 GiB for activations and the rest, before context.
• KV cache: the KV cache is the part of the footprint that grows with the conversation, and on this model FP16 KV costs 64 KiB per token, which is about 0.5 GiB at 8K, 2 GiB at 32K and about 6.3 GiB at 100K. The hybrid attention backbone keeps that lower than a normal 27B, but it is not free.
• Vision projector: budget about 0.9 GiB extra when images are in play. BONSAI_MMPROJ_CPU=1 keeps it in system RAM instead, which is slower for image prompts and free for text.

The demo sizes context to your RAM rather than letting you OOM. In particular, it uses this ladder: 8,192 tokens up to 11 GB of RAM, 16,384 up to 23 GB, 32,768 up to 35 GB, 65,536 up to 71 GB, and 131,072 above that for the 27B. The full 262,144 needs an explicit BONSAI_CTX=262144, and the scripts will not do that silently.

Peak memory in that table is our arithmetic, not a vendor measurement, and we ran the numbers the same way we do for every machine we cover: the measured file size, plus Prism ML's ~1.2 GiB overhead, their published per-token KV cost and the ~0.9 GiB projector. The budget side follows the same rule our recommender applies everywhere, roughly 70 percent of unified memory at 32 GB rising to 85 percent at 128 GB and up, which leaves room for macOS and your other apps. The full reasoning is in how much VRAM an LLM needs.

That said, two caveats hang on the 16 GB row. It works because the weights are 5.54 GiB, and it stops working if you insist on PQ2_0 plus a 131K context plus the projector, which lands near 16.8 GiB. And a 2-bit class file is not a free lunch in general, which is why our standing position on that class of quantization is on the record in why we never recommend 2-bit quants. Bonsai 2 is the exception worth testing precisely because it is ternary with FP16 scales, and because Prism ML publishes the failure cases of conventional 2-bit builds next to their own numbers.

Speed and quality: vendor numbers, no independent reproduction

In fact, everything in this section is Prism ML's own measurement, from the model card and the whitepaper. We have not reproduced it, and we found no third-party run of Bonsai 2 on Apple Silicon yet. Treat these as claims with a source, not as facts you can plan around.

One discrepancy is worth flagging, specifically because it shows how fresh these numbers are. The cross-platform throughput table in the same model card lists the M5 Pro at 28.1 tok/s decode and 387 tok/s pp512 on the current stack, and labels the 28.7 and 393 pair as pending re-measurement. Use the lower pair when you plan. Notably, Prism ML measures the M5 Pro at 27.5 W on the GPU rail and 34.1 W across CPU and GPU. That is why a 27B model runs on battery at all. Quality, same caveat. Bonsai 2 27B averages 84.78 across 14 thinking-mode benchmarks, against 86.32 for Qwen3.8-27B FP16, which is 98.2 percent retained. For scale, a conventional IQ2_XXS build of the same base model scores 72.59 at 9.4 GB. The category breakdown matters more than the average. For example, math is 96.57 against 97.06 for FP16 and coding is 89.42 against 89.07, so the losses concentrate in knowledge and reasoning at 79.86 against 85.55, plus vision at 66.19 against 71.36.

A community benchmark table exists, and you should read it carefully before quoting it. Meanwhile, the vendor table above is the one to plan against. It lists an Apple M5 Max 48 GB at 816 t/s pp512 and 45.8 t/s tg128 on llama.cpp Metal. Those are previous generation Ternary-Bonsai-27B numbers, not Bonsai 2, and the generation is a different file format with different kernels. Specifically, the same table lists an M5 Pro 64 GB at 130 t/s pp512 and 26.5 t/s tg128 on Metal. Useful as a sanity check on Metal, and useless as a Bonsai 2 measurement.

If you want the math run on your own machine, the LLM memory calculator sizes your unified memory to the models that fit, and the budget rule differs from a graphics card.

FAQ

Does Bonsai 2 27B run in Ollama or LM Studio?

No, and the reason is structural. Both sit on llama.cpp, and as a result stock llama.cpp cannot run any Bonsai 2 band. PTQ1_0 and PQ2_0 are rejected as unknown tensor types 142 and 143. Q2_0 loads without an error and produces gibberish. Until llama.cpp#29058 and the Hadamard transform land upstream, the PrismML fork is the only runtime.

How much RAM do I need for Bonsai 2 27B on a Mac?

Budget 7.9 to 8.6 GiB for the PTQ1_0 band at 16K context, including KV cache and about 1.2 GiB of overhead, and 10.8 GiB for PQ2_0 at 32K. In other words, a 16 GB Mac handles the small band. For 131K context, plan on 16.8 GiB with an FP16 KV cache or 11.1 GiB with BONSAI_KV4=1.

Why does the file need a fork if the architecture is unchanged?

The architecture is Qwen3.8-27B unchanged. However, the weights are not. Each matrix is rotated blockwise by a Hadamard transform before the ternary assignment, and the runtime has to apply the matching transform to activations. That transform is not upstream yet, so a build without it either refuses the file or, in the Q2_0 case, runs it wrong.

Is the 5.9 GB file the same as a 2-bit quantization?

No, and the distinction is the point. The weights are ternary, one of minus one, zero or plus one, with one FP16 scale per 128 weights. That is a true 1.75 bits per weight in the dense packing, and Prism ML reports 98.2 percent of FP16 quality against 84.1 percent for a conventional IQ2_XXS build. Whether that holds up outside the vendor's own evaluation is still an open question.

Can I use the MLX pack instead of the llama.cpp fork?

Yes, although with a different caveat. In contrast to the GGUF path, the MLX 2-bit pack is plain safetensors and does not need the llama.cpp fork, but it does not load in an ordinary MLX loader either. It ships a bundled loader in runtime/ that applies the required transforms, and that loader is text-only. Vision goes through Prism ML's demo with stock mlx-vlm.

Bottom line

Bonsai 2 27B is the first 27B-class model where the memory question has a comfortable answer on a 16 GB Mac and the runtime question has a bad one. As a result, the buying advice splits in two. If you have already installed Ollama or LM Studio and expected this to be a pull and a prompt, it will not be. If you are willing to run one extra command, ./setup.sh from the demo repo, you get a 5.54 GiB 27B with 262K context, vision and tool calling on hardware you already own. The vendor reports 18 to 47 tok/s depending on the chip.

Size the machine for the band you actually want, not for the 5.9 GB headline: 8.6 GiB of peak footprint on a 16 GB Mac, 11 to 13 GiB if you want PQ2_0 with a working context, and 24.8 GiB only if you insist on the full 262K with FP16 KV. And keep the upstream issue bookmarked. The day types 142 and 143 merge, the answer to "does it run on my Mac" becomes a memory question again. At that point this article needs a rewrite rather than an update.

Sources and method

Every figure here comes from a primary source we opened ourselves, and the memory table is our arithmetic, labelled as such. Found an error? Contact us: our editorial standards are on the about page.

• prism-ml/Ternary-Bonsai-2-27B-gguf and its Hugging Face API file listing: exact byte sizes for PTQ1_0, PQ2_0, mmproj Q8_0 and F16, read September 18, 2026.
• Bonsai 2 27B model card: architecture, bit widths, benchmark table, vendor throughput numbers, generation parameters.
• prism-ml/Ternary-Bonsai-2-27B-mlx-2bit: 8.60 GB safetensors pack, 7.67 GB language model plus a 0.92 GB vision tower, bundled runtime/ loader.
• PrismML-Eng/Bonsai-demo MODEL-FORMATS.md: the rotated-basis quote, the missing works-everywhere band, the Q2_0 silent-failure warning.
• PrismML-Eng/Bonsai-demo README and AGENTS.md: context tiers, overhead and projector budgets, upstream status.
• Bonsai-demo KV-CACHE.md: 64 KiB per token FP16, about 18 KiB per token with BONSAI_KV4=1, ~6.3 GiB versus ~1.8 GiB at 100K.
• PrismML-Eng/llama.cpp releases: prism-b10685-7dffb15 (September 15, 2026) carries the macOS arm64 and KleidiAI archives; prism-b10687-5d80cff (September 17, 2026) is a partial asset set.
• ggml-org/llama.cpp#29058: feature request for types 142 and 143, opened September 18, 2026, open and uncommented.
• Ternary-Bonsai community benchmarks: Apple results for the previous generation Ternary-Bonsai-27B, including the M5 Max 48 GB at 816 t/s pp512 and 45.8 t/s tg128.
• prism-ml/Ternary-Bonsai-2-27B-gguf-dev: Ternary-Bonsai-2-27B-Q2_0-prism-fork-required.gguf at 7,626,008,928 bytes, testing only.