How Much VRAM Do You Need for Local LLMs? (2026)

VRAM is the one spec that decides what you can run locally, and it is the one most guides get wrong by counting "loads" as "runs." The rule is simple: a model needs its quantized weights plus context cache in fast memory, or it slows to a crawl. This is the cheat sheet we use internally, refreshed on September 3, 2026, with the tier-by-tier table and the traps.

TL;DR: At Q4_K_M, budget 0.6 GB per billion parameters for weights, plus 1-4 GB for context. 8GB runs 7B-8B, 12GB runs 12B, 16GB runs 14B and tight 26B MoE, 24GB runs 27B-32B, 32GB runs 35B MoE, 48GB+ runs 70B. Anything bigger wants 96GB. Offloading to system RAM is not a workaround; it is a slowdown of 3-10x.

Want your own number instead of a table? The VRAM calculator sizes a card to real picks, the GPU VRAM calculator answers card by card, VRAM per model size starts from the model, and RAM vs VRAM covers Apple Silicon against a discrete card.

What is the only formula you need?

VRAM needed = (parameters x quant size) + context cache + OS overhead.

At Q4_K_M, weights cost about 0.6 GB per billion parameters. A 7B model is about 4.5 GB. A 27B is about 16-17 GB. Context cache adds roughly 128-192 KB per token for dense models at fp16, so a 32K context adds 4-6 GB on a mid-size model. Windows and your desktop take another 1-2 GB. For exact per-model quants rather than the formula, our quant comparison tool lists the real file sizes, and the official builds on orgs like Qwen and Meta's Llama hub are the ground truth we check them against.

Mixture-of-experts models change the speed math, not the memory math. A 122B-A10B MoE still needs memory for all 122B parameters; only its decode speed behaves like a smaller model. We ran the formula against every model in our catalog and it holds within a few percent at Q4, which is exactly why "loads" and "runs" are not the same word here.

How much VRAM does each tier need?

Mac unified memory maps directly onto this table. A 16GB Mac behaves like a 12GB GPU after macOS overhead. A 24GB Mac runs the same models as a 24GB GPU, slower per byte but with the same capacity. Every example pick above has its own model page, such as GPT-OSS 20B, Qwen 3.8 27B, and Llama 3.3 70B.

What is each tier good for?

• 8GB is real local AI now: chat, coding help, and agents with 7B-8B models. See the 8GB picks.
• 12GB unlocks the 12B dense class, the biggest quality jump at the low end.
• 16GB adds 14B and the first 20B-class MoE. The value tier. See the 24GB and 16GB Mac guides.
• 24GB is the enthusiast sweet spot: 27B dense or fast 20B MoE.
• 32GB is the 35B MoE tier, where sparse models start to rival frontier quality.
• 96GB is the 120B MoE tier, the largest models that run on a single consumer or workstation machine.

What is the offload trap?

Every "this GPU runs a 70B" claim is CPU offload. The model's weights sit in slow system RAM and the GPU fetches them per token. An RTX 4090 "running" a 70B produces about 1.6 tok/s est. An RTX 5090 "running" a 122B MoE produces about 15 tok/s est., a third of what it does fully resident. Offload is a demo, not a workflow.

The honest test: is the model fully inside fast memory with room for context? If not, the tok/s number in the listing is the offloaded one, and it will feel broken within the first conversation. Consider the 4090-and-70B claim for a second: at 1.6 tok/s est., a short answer takes half a minute, and that is not a workflow anyone keeps past the first afternoon.

How much memory does context cost?

VRAM guides often quote weights only, but a long conversation is not free and the bill scales with both length and architecture. A dense 27B model at 128K context can use 24+ GB of KV cache at fp16, which is more than its own weights, so a machine that "fits" the model can still fail the workload. Hybrid linear-attention models (Qwen 3.5, Qwen 3.6, Qwen 3-Next) cache KV only on their sparse full-attention layers, and as a result their long-context memory cost runs 4-13x lower than a dense model of the same size. If long documents are your use case, that architecture difference matters as much as raw VRAM; in fact it is often the deciding factor.

How do you sanity-check any VRAM claim?

When a page tells you a model runs on your machine, three questions separate an honest answer from a listing. Does the claim name the quant, because a model at Q4 and the same model at Q2 are different products with different quality? Does it distinguish loading from running, which you can test by asking whether every weight sits in fast memory with room left for context? And does it say anything about speed, because a fit verdict without tokens per second is only half the story?

If any answer is missing, you can reproduce the math yourself with the formula at the top of this page, or run the same check we run on our hardware fit checker. The whole point of this cheat sheet is that VRAM decisions should be arithmetic, not faith.

FAQ

Is 8GB VRAM enough for local LLMs?

Yes, for 7B-8B models. Qwen 3 8B and Qwen 3.5 4B run well on an RTX 4060. It is not enough for 12B or for long contexts.

How much VRAM for a 70B model?

About 42 GB for Q4_K_M weights, so 48GB is the floor. A 64GB+ Mac Studio runs it; a 24GB GPU does not. Anything claiming otherwise is offloading.

Does more VRAM make a model faster?

No, once the model fits. Extra VRAM beyond the weights and context does not speed up decode; memory bandwidth does. More VRAM lets you run a bigger model or a longer context, at the same per-token speed.

GPU VRAM or Mac unified memory?

Both work from the same table. A GPU is faster per GB of bandwidth; a Mac scales to much larger memory pools cheaply. For 70B-120B models, a Mac Studio is usually the cheaper path. For 8B-32B at maximum speed, a GPU wins.

___

The tier table and every example speed here come from the ModelFit engine, refreshed September 3, 2026, with estimates labeled est. How we work: about ModelFit. Corrections welcome: contact.