NVIDIA Nemotron 3.5 Bolt on a Mac: 22GB Needs a 36GB Mac, No Metal Yet

NVIDIA Nemotron 3.5 Bolt 74B-A9B, released on Hugging Face on October 15, 2026, is a 74B vision-language model that NVIDIA compressed and quantized "to fit in 24GB of memory" (NVIDIA model card, 2026). On a Mac, the verdict has two parts. First, it does not run out of the box today: Ollama states that Apple Silicon and Metal are not currently supported, and llama.cpp crashes on Metal until a one-line kernel fix lands. Second, once it does run, the realistic floor is a 36GB Mac, not a 24GB one, because the 22,018,326,528-byte GGUF plus its vision projector does not fit the share of unified memory macOS lets a model use on smaller machines.

What is Nemotron 3.5 Bolt 74B-A9B?

Nemotron 3.5 Bolt is an agentic vision-language model with 74B total parameters and 9B active per token. It is a Mamba-2 and Transformer hybrid Mixture-of-Experts with a C-RADIO v4-H vision encoder, accepts text, images and video, and supports up to 256K tokens of context (NVIDIA model card, 2026).

NVIDIA's own description is blunt about the target: the model is "designed to run everyday agentic tasks locally, using a native 2-bit quantization to fit in 24GB of memory and run alongside other applications." The card names RTX Spark (64GB and 128GB) and RTX discrete GPUs (32GB and up) as deployment hardware. It lists Windows and Linux, Ada Lovelace and Blackwell. No Mac appears anywhere on the card.

The checkpoint's config shows why a 74B model can be this small in memory terms. Of its 88 blocks, 40 are Mamba layers, 40 are MoE layers and only 8 are attention layers, with 2 key-value heads of dimension 128 (config.json, 2026). Few attention layers means a small KV cache, which matters for the RAM math below.

Can a Mac run Nemotron 3.5 Bolt today?

Not without patching llama.cpp yourself. The Ollama library page for nemotron-3.5-bolt says it plainly: "Apple Silicon / Metal is not currently supported" (Ollama library, 2026). Ollama is the easiest path for most Mac users, so for them the answer today is no.

On llama.cpp, support for the new architecture merged on October 15 in pull request 30520 and shipped in the v0.7.0 release the same day. On a Mac, though, the model loads and then crashes on the first request. Issue 30610, opened on an M1 Max with 32 GB, traces the crash to a Metal MoE kernel that is not compiled for 9 routed experts per token, which is exactly what Bolt uses. A commenter proposed adding the missing kernel variant, and the reporter confirmed that with it the IQ2_XXS model "now answers normally", text only. As of October 16 that issue is still open and no fix has merged.

So the practical status is: Linux and Windows on NVIDIA hardware work now, Macs wait for a llama.cpp fix and then for Ollama to pick it up. When the fix ships, the memory question below decides which Macs qualify.

How much unified memory does Nemotron 3.5 Bolt need on a Mac?

The smallest Mac that holds the model plus its vision projector inside ModelFit's memory budget is 36GB. A 24GB Mac is out, and a 32GB Mac only fits the text model with almost nothing left for context. Here are the real file sizes, summed from the Hugging Face API file listings in decimal megabytes:

Sources: the GGUF repo and the LiLiCorr GGUF repo. The Ollama tags ship the same three blobs at the same byte counts, which is why the library page lists the model at 25GB.

ModelFit does not give a model all of a Mac's RAM. macOS caps how much unified memory the GPU can wire, so the engine budgets 70% of RAM at 32GB and below, rising linearly to 85% at 128GB. That is the same rule behind every pick on the homepage wizard and in the RAM guide for local LLMs. Applied to Bolt:

The "24GB" in the model name describes the deployment footprint NVIDIA aimed for on its own hardware, not what a 24GB Mac can hold. On a 24GB Mac the budget is about 16,800 MB, more than 5 GB short of the weights alone.

How big is the KV cache, and why do NVIDIA's own recipes ask for 28GB and 32GB?

The KV cache is small for a model this size, because only 8 of the 88 blocks use attention. With 2 key-value heads of dimension 128 at 16-bit precision, each token costs 8 layers x 2 (key and value) x 2 heads x 128 x 2 bytes, or 8,192 bytes. A 131,072-token context therefore adds about 1,074 MB, and the full 262,144-token window about 2,147 MB. That arithmetic comes from the published config, not from a run on hardware. It also leaves out the Mamba state and the runtime's compute buffers, which are real and not published.

NVIDIA's model card is the better guide to the total. It says the architecture search targeted "a deployment footprint below 24 GB at batch size 1 and 128K context", but the llama.cpp recipes it actually publishes are larger:

• 28GB recipe: 65,536-token context, q8_0 KV cache, up to 6 draft tokens, for RTX Spark 64GB or a 32GB+ discrete GPU.
• 32GB recipe: 262,144-token context, up to 7 draft tokens, for RTX Spark 128GB or a 48GB+ discrete GPU.

Map those onto a Mac and the picture is consistent with the table above. A 36GB Mac (25,425 MB budget) sits below even the 28GB recipe, so plan on a short context and probably no drafter. A 48GB Mac (34,800 MB budget) clears the 32GB recipe, which is why 48GB is the tier where Bolt behaves the way NVIDIA documents it. You can rerun this math for any context length in the LLM memory calculator.

Is a 2-bit model worth running? What IQ2_XXS costs here

This is NVIDIA's own mixed 2-bit recipe, trained for, not a generic 2-bit quant squeezed out of a finished model. ModelFit never recommends 2-bit quants as a rule, because post-hoc 2-bit compression of a model trained at 16 bits usually breaks reasoning in ways a file-size chart hides. Bolt is a different case, and the difference is worth stating exactly.

According to the model card, NVIDIA quantized all experts, routed and shared, to IQ2-XXS at 2.0625 bits per element, then used quantization-aware distillation (QAD) to recover accuracy. Not everything is 2-bit: the Mamba layers stay in NVFP4 and the embedding and output layers in Q8_0. The published benchmarks are run on this IQ2 build, not on a 16-bit parent, so the 2-bit cost is already inside the scores below.

There are two honest caveats. First, every score on the card comes from NVIDIA; no independent evaluation of the IQ2 build exists yet. Second, the card's own table shows weak spots, so the 2-bit trade is not free everywhere.

How does Bolt score against the models a Mac already runs?

On NVIDIA's own table, Bolt's IQ2 build leads Qwen3.6-35B-A3B at BF16 on most agentic and reasoning tests. The figures below are copied from the model card (2026):

Bolt trails on AA-Omniscience accuracy, where Gemma4 26B-A4B scores 22.17 on the same table, and roughly ties on PinchBench. For a cross-model view against cloud anchors, the ModelFit benchmark tracks the local models it can source. Bolt is not on it yet because no third-party score for it exists.

Where does Bolt come from? The Nemotron 3.5 Super VL parent

Bolt is a compressed child of Nemotron 3.5 Super VL 120B-A12B. NVIDIA used its Star Elastic architecture search to cut the parent down, fixing the expert top-K at 9, "down from 22 in the parent model" (model card, 2026). After the vision adapter was re-attached and fine-tuned, the model went through SFT, RL and finally the 2-bit quantization.

The size gap explains the whole project. The parent's BF16 repository holds 248,913,061,416 bytes of safetensors, about 249 GB. Bolt's own BF16 checkpoint is 149,774,331,136 bytes, about 150 GB, and the shipped 2-bit GGUF is about 22 GB. That last step is what moves a 74B model from a workstation to a laptop-class memory pool.

What about the third repo, NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr? It is not another model to chat with. Its card describes a 1.113B-parameter speculative-decoding drafter that "is not a standalone language model": it proposes draft tokens that Bolt verifies. That checkpoint targets vLLM on Blackwell; its GGUF twin is the 998 MB drafter in the size table above.

Which Ollama tags exist for Nemotron 3.5 Bolt?

Three tags exist, and today all three point to the same build: latest, 74b-a9b and 74b-a9b-iq2. Each manifest answered 200 on the Ollama registry on October 16, and each bundles the main GGUF, the vision projector and the drafter. On a Linux or Windows machine with a supported NVIDIA GPU, the command is:

ollama run nemotron-3.5-bolt:74b-a9b-iq2

On a Mac, the same Ollama page says Metal is not currently supported, so do not expect this command to work there until that note changes.

How does a Mac compare with an RTX Spark laptop for Bolt?

Bolt was built for the RTX Spark launch, and NVIDIA's 28GB and 32GB recipes map directly onto the 64GB and 128GB RTX Spark configurations. A Mac with the same unified memory can hold the same files once llama.cpp's Metal path is fixed; the difference today is software support, not capacity. Our RTX Spark versus Mac comparison covers the rest of the model lineup on both machines.

If you already own a Mac and want a local agent model that runs now, check what your exact machine supports with the wizard, or from a terminal with npx @wecko-ai/modelfit. Both use the same memory budget as the tables in this article.

How were these numbers put together?

The methodology is simple and fully reproducible. File sizes are byte counts summed from the Hugging Face API file listings and the Ollama registry manifests on October 16, 2026, converted to decimal megabytes. Memory budgets come from ModelFit's engine rule, for example 70% of 32GB, which gives 22,400 MB. Benchmark scores are copied from NVIDIA's model card, such as the GPQA Diamond row. Nothing here was run on a Mac or on RTX Spark hardware; ModelFit runs no benchmarks. More on how the site works is on the About page.

FAQ

Can I run Nemotron 3.5 Bolt on a 24GB Mac?

No. ModelFit's engine budgets 70% of unified memory at 24GB, about 16,800 MB, while the main GGUF alone is 22,018 MB. The "24GB" in the name refers to the footprint NVIDIA targeted on its own GPUs, where nearly all memory is usable by the model. On a Mac, the smallest tier that holds the model plus its vision projector is 36GB.

Does Nemotron 3.5 Bolt work in Ollama on a Mac?

Not yet. The Ollama library page states that Apple Silicon and Metal are not currently supported. In llama.cpp the model loads on Metal but crashes on the first request because a MoE kernel variant for 9 experts per token is missing. A one-line fix was confirmed in the issue thread on an M1 Max but has not merged as of October 16.

Is the 2-bit quantization going to ruin quality?

Less than a generic 2-bit quant would. NVIDIA trained Bolt for this format with quantization-aware distillation, keeps Mamba layers in NVFP4 and embeddings in Q8_0, and reports its benchmarks on the IQ2 build itself. The caveat is that every score so far comes from NVIDIA. Independent evaluations of the IQ2 build have not been published yet.

How much memory does the full 256K context need?

NVIDIA's own llama.cpp recipe for the full 262,144-token context is the 32GB recipe, aimed at RTX Spark 128GB or a 48GB+ GPU. On a Mac that points to 48GB, where the engine budget is about 34,800 MB. The KV cache itself is small, roughly 8,192 bytes per token at 16-bit, because only 8 of the 88 blocks use attention.

Sources

• NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF model card (NVIDIA, 2026)
• NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB checkpoint and config.json (NVIDIA, 2026)
• NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr and its GGUF (NVIDIA, 2026)
• NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16 (NVIDIA, 2026)
• NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 (NVIDIA, 2026)
• Ollama library: nemotron-3.5-bolt (Ollama, 2026)
• llama.cpp pull request 30520 and issue 30610 (ggml-org, 2026)