VRAM per Model Size
Real GGUF file sizes by size class, the smallest GPU tier that fits each Q4 build, and what 32k of context adds on top of the weights.
How much VRAM does an NB model need? An NB model is one of N billion parameters: budget about 0.6 GB per billion parameters at Q4_K_M for the weights alone, against a usable budget of roughly 90% of the card. On the calculator's default 12 GB card that is ~11 GB of weights, which clears every 7B-9B Q4 build in the table below. Context is charged separately: the KV cache grows with every token you keep, which is why the same model can need a bigger card at 32k than at 8k. Real GGUF sizes follow.
With 12 GB of VRAM, ModelFit budgets about 11 GB for the model and comfortably runs local LLMs up to ~12B parameters at Q4. The best single pick is Qwen3.5 9B Instruct. Estimates use a representative 12 GB card (RTX 3060).
Tokens/sec are ModelFit estimates from memory bandwidth and model size, not measured benchmarks. Ollama commands are registry-verified.
Real file sizes by size class
One representative GGUF per size class from the 30 probed repos. Every size in the table is the actual file size on Hugging Face, not a rule-of-thumb estimate; a quant that the repo does not ship renders as n/a.
| Model | Params | Q4_K_M | Q5_K_M | Q8_0 | KV at 32k |
|---|---|---|---|---|---|
| Gemma 3 1B Instruct | 1B | 0.8 GB | 0.8 GB | 1 GB | 3.5 GB |
| Qwen3.5 4B Instruct | 4B | 2.8 GB | 3.2 GB | 4.3 GB | 1 GB (exact config) |
| Qwen3.5 9B Instruct | 9B | 5.8 GB | 6.6 GB | 9.1 GB | 1 GB (exact config) |
| Phi-4 14B | 14B | 8.4 GB | 9.9 GB | 14.5 GB | 6 GB |
| Qwen3.5 27B Instruct | 27B | 16.8 GB | 19.5 GB | 27.1 GB | 2 GB (exact config) |
| Qwen3.5 35B-A3B Instruct | 35B (3B active) | 20.8 GB | 24.1 GB | 35.2 GB | 0.6 GB (exact config) |
| Qwen3-Next 80B-A3B | 80B (3B active) | 45.2 GB | 52.9 GB | 79 GB | 0.8 GB (exact config) |
Sizes are real GGUF file sizes probed from Hugging Face repos on 2026-09-02 - for example bartowski/Phi-4-GGUF. KV at 32k is an fp16-cache estimate from each model's catalog entry. Updated 2026-09-18.
Smallest GPU tier per size class
For each row above, the smallest real card tier whose 90% usable budget fits the Q4_K_M file, and whether 32k of context pushes the total to the next tier.
| Model | Q4_K_M weights | Smallest card that fits | At 32k context |
|---|---|---|---|
| 1B | 0.8 GB | fits a 8 GB card | fits the same card |
| 4B | 2.8 GB | fits a 8 GB card | fits the same card |
| 9B | 5.8 GB | fits a 8 GB card | fits the same card |
| 14B | 8.4 GB | fits a 12 GB card | pushes up to a 24 GB card (KV cache included) |
| 27B | 16.8 GB | fits a 24 GB card | fits the same card |
| 35B | 20.8 GB | fits a 24 GB card | fits the same card |
| 80B | 45.2 GB | fits a 96 GB card | fits the same card |
Fit checks apply the 90% usable-VRAM budget to real card tiers; the KV column is the same fp16 estimate as the table above. KV-cache figures assume an fp16 cache, the llama.cpp/Ollama default. Standard GQA models use a size-class estimate (8 KV heads x 128 head dim class); hybrid linear-attention models (Qwen3.5/3.6, Qwen3-Next) use the exact per-token cost from their published config, since only their sparse full-attention layers cache KV. A q8_0 KV cache roughly halves either figure. Estimates, not measurements.
Why two models of the same size need different VRAM
Quantization is the first lever: Phi-4 14B weights are 8.4 GB at Q4_K_M but 14.5 GB at Q8_0, so the same model costs nearly twice as much before a single token of context. The second lever is the KV-cache layout: classic GQA models fall into a size-class estimate of 128-384 KB per token, while hybrid linear-attention models such as the Qwen3.5 and Qwen3-Next rows above carry an exact per-token cost from their published config, because only their sparse full-attention layers cache KV. Context length is the third lever and it multiplies that per-token cost linearly, which is why the KV column in this table is charged separately from the weights. Add all three and a 27B from one family can need a different card than a 27B from another.
Frequently asked questions
How much VRAM does a 7B model need?
Weights at Q4_K_M cost about 0.6 GB per billion parameters, so a 7B model comes in near 4.2 GB. In the probed set behind this page, Mistral 7B is 4.1 GB and DeepSeek-R1-Distill 7B is 4.4 GB at Q4_K_M, with the Q8_0 mirrors at 7.2 and 7.5 GB. An 8 GB card offers roughly 7 GB of usable budget after the 90% overhead rule, so a 7B at Q4 fits comfortably, and the Q8 build fits only with a short context. That is why the 7B-9B class rows in the tier table land on an 8 GB card.
How much VRAM does a 70B model need?
A dense 70B at Q4_K_M needs about 42 GB of weights by the 0.6 GB per billion rule, and the KV cache is charged separately: 10 GB more at 32k context on the GQA size-class estimate. Nothing below the 96 GB tier clears 52 GB of total load - the 32 GB tier only has about 29 GB usable - which is why a 70B-class build starts at the 96 GB tier (about 86 GB usable). The closest probed row on this page, Qwen3-Next 80B-A3B (a mixture-of-experts model with 3B active), loads 45.2 GB at Q4_K_M and needs the same tier.
Does Q8 need twice the VRAM of Q4?
Not exactly, and the probed files say so: across the seven size-class rows, the Q8_0 file runs from 1.3x (Gemma 3 1B Instruct) to 1.7x (Qwen3-Next 80B-A3B) of the Q4_K_M size, so "twice" overstates the typical case. Q4_K_M and Q8_0 are both partial quantizations with different group sizes and headroom, and the gap tends to widen as models grow. The safe planning range for a Q8 mirror build is about 1.3x to 1.7x the Q4 size, and the KV cache is unchanged, so Q8 also leaves less room for context.
Why do two 8B models from different families differ in VRAM?
A size class is a family of files, not one file. Qwen3 8B is 4.7 GB at Q4_K_M while Llama 3.1 8B is 4.6 GB - both 8B, both Q4, different bytes, and the same at Q8_0 (8.1 vs 8 GB). The difference comes from vocab and embedding size, layer count and KV-head layout, and small architectural extras baked into the GGUF. Context widens the gap further because the KV cache is not proportional to parameters either: a 9B with a classic layout estimates 128 KB per token in the size-class model, while the hybrid linear-attention Qwen3.5 9B caches 32 KB per token from its published config.
Go deeper
The other direction: start from the card, get the best models that fit.
Weights plus KV cache plus the real memory budget of a Mac or a card.
Per-card answers across the NVIDIA lineup, tier by tier.
When system memory can back a model and when only VRAM will do.
Unified memory, single card and multi-GPU rigs in one tool.
Every quant per model and memory tier, with fit verdicts.
More surfaces: all model pages · can I run it · best hardware for a model · how much VRAM for LLMs · open dataset · hardware stats
The weekly local-AI refresh
New open-weight models, real Apple Silicon benchmarks, and the one model worth running on your Mac this week. Free, one email a week, unsubscribe anytime.
By subscribing you agree to our Privacy Policy and to receive the weekly email. Unsubscribe anytime.