LLM Memory Calculator
Weights, context cache, and the memory your machine will actually hand over. Enter a machine and see the single best model it runs.
How much memory does a local LLM need? Start with 0.6 GB per billion parameters at Q4 for the weights, roughly double that at Q8, then add the KV cache your context keeps alive, and finally leave room for the runtime. ModelFit budgets 70% to 85% of a Mac's unified memory and 90% of a graphics card. A 16GB Mac therefore has about 11.2 GB of model budget, which the engine turns into one best pick and the alternatives that fit beside it.
With 16 GB of unified memory, ModelFit budgets about 11 GB for the model and comfortably runs local LLMs up to ~12B parameters at Q4. The best single pick is Qwen3.5 9B Instruct.
Tokens/sec are ModelFit estimates from chip bandwidth and model size, not measured benchmarks. Ollama commands are registry-verified.
What the weights cost
The first term of the calculation, at the two quantizations that cover most Ollama pulls. Q8 is close to the original weights, Q4_K_M is the usual default and roughly half the size.
| Parameters | Weights at Q4_K_M | Weights at Q8 |
|---|---|---|
| 3B | ~1.8 GB | ~3.2 GB |
| 7B | ~4.2 GB | ~7.4 GB |
| 9B | ~5.4 GB | ~9.5 GB |
| 14B | ~8.4 GB | ~14.8 GB |
| 27B | ~16.2 GB | ~28.6 GB |
| 32B | ~19.2 GB | ~33.9 GB |
| 70B | ~42 GB | ~74.2 GB |
Weights only, before the KV cache and runtime overhead. Estimates from ModelFit's catalog of 106 local models, not measured benchmarks.
What context adds
The second term, and the one that turns a comfortable fit into an out-of-memory error. Each row is a real catalog entry, and the per-token cost comes from its own configuration.
| Model | Weights | 8k context | 16k context | 32k context | 64k context | 128k context |
|---|---|---|---|---|---|---|
| Qwen3.5 4B Instruct (4B) | ~3.5 GB | ~0.3 GB | ~0.5 GB | ~1 GB | ~2 GB | ~4 GB |
| Qwen3.5 9B Instruct (9B) | ~7 GB | ~0.3 GB | ~0.5 GB | ~1 GB | ~2 GB | ~4 GB |
| Qwen3.5 27B Instruct (27B) | ~16 GB | ~0.5 GB | ~1 GB | ~2 GB | ~4 GB | ~8 GB |
| Llama 3.3 70B Instruct (70B) | ~42 GB | ~2.5 GB | ~5 GB | ~10 GB | ~20 GB | ~40 GB |
KV-cache figures assume an fp16 cache, the llama.cpp/Ollama default. Standard GQA models use a size-class estimate (8 KV heads x 128 head dim class); hybrid linear-attention models (Qwen3.5/3.6, Qwen3-Next) use the exact per-token cost from their published config, since only their sparse full-attention layers cache KV. A q8_0 KV cache roughly halves either figure. Estimates, not measurements.
The budget you actually have
Unified memory is one pool shared with macOS, so its usable share is lower than a discrete card's. Same number, two very different answers.
| Memory | Usable on an Apple Silicon Mac | Usable on a discrete GPU |
|---|---|---|
| 16 GB | ~11.2 GB | ~14.4 GB |
| 24 GB | ~16.8 GB | ~21.6 GB |
| 32 GB | ~22.4 GB | ~28.8 GB |
| 48 GB | ~34.8 GB | ~43.2 GB |
| 64 GB | ~48 GB | ~57.6 GB |
| 128 GB | ~108.8 GB | ~115.2 GB |
The Apple column assumes the default wired-memory limit; owners of 96GB machines and above routinely raise it. The tier-by-tier model matrix lives in the how much RAM guide, and the same numbers drive the calculator.
Frequently asked questions
How do I calculate the memory a local LLM needs?
Add three things: the weights, the KV cache, and the runtime overhead. At Q4_K_M the weights cost about 0.6 GB per billion parameters, so a 14B model is roughly 8.4 GB. The KV cache holds your conversation and grows with context: from about 112 KB per 1,000 tokens on a small model to over 320 KB on a 70B model, halved if you quantize the cache to q8_0. The runtime and OS need the remainder, which is why ModelFit budgets 70% to 85% of unified memory and 90% of a graphics card rather than the whole figure.
Why can I not use all of my memory for the model?
On a Mac the GPU shares unified memory with macOS and every open application, so ModelFit budgets 70% of it up to 32GB and about 85% at 128GB and above, where the wired-memory limit is usually raised. That leaves a 16GB Mac with roughly 11.2 GB of usable model budget. On a discrete card the driver context and the display take a smaller cut, so the budget is 90% of VRAM.
How much memory does context add?
The KV cache scales linearly with context length, and it is the figure people forget. As a worked example from ModelFit’s catalog, Qwen3.5 27B Instruct costs about 64 KB per token of context, which is roughly 1 GB at 16k tokens, 2 GB at 32k and 4 GB at 64k on top of its 16 GB of weights. Llama 3.3 70B sits in a heavier size class at about 320 KB per token, so 64k of context adds around 20 GB to a model that already needs 42 GB. Hybrid linear-attention models such as Qwen3.5 cache only a fraction of what a classic attention layout would.
Does quantizing the KV cache help?
Yes. A q8_0 KV cache roughly halves the per-token cost with a small quality cost, which is often the cheapest way to run a long context on fixed hardware. It is the second lever after the weights themselves: quantizing the weights from Q8 to Q4 cuts the model roughly in half, and quantizing the cache does the same for the context side of the bill.
Unified memory or dedicated VRAM?
Unified memory is one pool shared by the CPU and GPU, which is why an Apple Silicon machine can hold a model far larger than its graphics share would suggest, and why the usable budget is lower as a percentage. A discrete card gives a higher share of its memory to the model but caps you at the memory on the card, unless you pool two or three cards. The calculator handles both modes, plus pooled rigs.
Go deeper
Choosing a graphics card instead? Start from the per-card answers.
Unified memory, single card and multi-GPU rigs in one tool.
Per-model tables, quantization ladders, and the context trap in prose.
What the arithmetic looks like on the most common configuration.
More surfaces: quantization comparison · can I run it · hardware for a model · model pages · open dataset · hardware stats · GPU hub · MacBook Air · Mac Studio · models for 16GB
The weekly local-AI refresh
New open-weight models, real Apple Silicon benchmarks, and the one model worth running on your Mac this week. Free, one email a week, unsubscribe anytime.
By subscribing you agree to our Privacy Policy and to receive the weekly email. Unsubscribe anytime.