Updated 2026-11-02

VRAM per Model Size

Real GGUF file sizes by size class, the smallest GPU tier that fits each Q4 build, and what 32k of context adds on top of the weights.

VRAM per billion params at Q4: ~0.6 GB
Usable for weights: ~90%
12 GB card, weights budget: ~11 GB
Size classes in the table: 7

How much VRAM does an NB model need? An NB model is one of N billion parameters: budget about 0.6 GB per billion parameters at Q4_K_M for the weights alone, against a usable budget of roughly 90% of the card. On the calculator's default 12 GB card that is ~11 GB of weights, which clears every 7B-9B Q4 build in the table below. Context is charged separately: the KV cache grows with every token you keep, which is why the same model can need a bigger card at 32k than at 8k. Real GGUF sizes follow.

Requirements calculator

GPU VRAM · 12 GB

Memory type

01

Memory.

Graphics card VRAM

VRAM

Top pick

Mellum2.1 12B-A2.5B ThinkingMellum · major

Runs wellB · 25

Fits in 12 GB VRAM with room to spare. Best for agentic coding, small machines on RTX 3060.

Footprint: 8.1 GB
Speed: est. 98 tok/s typical error 15%
First token: ~0.2s · Instant
Quant: Q4_K_M
Params: 12.1B

llama.cpp

Table 1Also fits

#	Model	Fit	Size	Load / 11 GB	Speed	First token
2	Qwen3.5 9B InstructQwen	Runs wellA · 35	9B / Q4_K_M	7 GB	est. 53 tok/stypical error 5%	~0.6s · Instant
3	Ornith 1.0 9BOrnith	Runs wellA · 48	9B / Q4_K_M	5.6 GB	est. 53 tok/stypical error 5%	~0.6s · Instant
4	Gemma 4 12BGemma	Runs wellB · 26	12B / Q4_K_M	8 GB	est. 35 tok/stypical error 15%	~0.9s · Instant
5	LFM2.5 8B-A1BLFM2	Runs wellA · 49	8.3B / Q4_K_M	5.5 GB	est. 139 tok/stypical error 15%	~0.2s · Instant
6	NVIDIA Nemotron 3.5 Bolt E4BNemotron	Runs wellB · 27	4B / Q4_K_M	7.9 GB	est. 67 tok/stypical error 5%	~0.6s · Instant
7	Gemma 4 E4B (Q8)Gemma	Runs wellA · 31	4.5B / Q8_0	7.5 GB	est. 53 tok/stypical error 5%	~3.5s · Deliberate
8	MiMo-V2.6-Distill-Qwen-9BMiMo	Runs wellA · 46	9B / Q4_K_M	5.8 GB	est. 51 tok/stypical error 5%	~0.6s · Instant

Tokens/sec are ModelFit estimates from memory bandwidth and model size, not measured benchmarks. What typical error means. Ollama commands are registry-verified.

See every model that fits 12 GB RTX 3060 deep dive

Table 2GGUF file sizes

Real file sizes by size class

One representative GGUF per size class from the 30 probed repos. Every size in the table is the actual file size on Hugging Face, not a rule-of-thumb estimate; a quant that the repo does not ship renders as n/a.

Model	Params	Q4_K_M	Q5_K_M	Q8_0	KV at 32k
Gemma 3 1B Instruct	1B	0.8 GB	0.8 GB	1 GB	3.5 GB
Qwen3.5 4B Instruct	4B	2.8 GB	3.2 GB	4.3 GB	1 GB (exact config)
Qwen3.5 9B Instruct	9B	5.8 GB	6.6 GB	9.1 GB	1 GB (exact config)
Phi-4 14B	14B	8.4 GB	9.9 GB	14.5 GB	6 GB
Qwen3.5 27B Instruct	27B	16.8 GB	19.5 GB	27.1 GB	2 GB (exact config)
Qwen3.5 35B-A3B Instruct	35B (3B active)	20.8 GB	24.1 GB	35.2 GB	0.6 GB (exact config)
Qwen3-Next 80B-A3B	80B (3B active)	45.2 GB	52.9 GB	79 GB	0.8 GB (exact config)

Sizes are real GGUF file sizes probed from Hugging Face repos on 2026-09-02 - for example bartowski/Phi-4-GGUF. KV at 32k is an fp16-cache estimate from each model's catalog entry. Updated 2026-11-02.

Table 3By GPU tier

Smallest GPU tier per size class

For each row above, the smallest real card tier whose 90% usable budget fits the Q4_K_M file, and whether 32k of context pushes the total to the next tier.

Model	Q4_K_M weights	Smallest card that fits	At 32k context
1B	0.8 GB	fits a 8 GB card	fits the same card
4B	2.8 GB	fits a 8 GB card	fits the same card
9B	5.8 GB	fits a 8 GB card	fits the same card
14B	8.4 GB	fits a 12 GB card	pushes up to a 24 GB card (KV cache included)
27B	16.8 GB	fits a 24 GB card	fits the same card
35B	20.8 GB	fits a 24 GB card	fits the same card
80B	45.2 GB	fits a 96 GB card	fits the same card

Fit checks apply the 90% usable-VRAM budget to real card tiers; the KV column is the same fp16 estimate as the table above. KV-cache figures assume an fp16 cache, the llama.cpp/Ollama default. Standard GQA models use a size-class estimate (8 KV heads x 128 head dim class); hybrid linear-attention models (Qwen3.5/3.6, Qwen3-Next) use the exact per-token cost from their published config, since only their sparse full-attention layers cache KV. A q8_0 KV cache roughly halves either figure. Estimates, not measurements.

Same size, different VRAM

Why two models of the same size need different VRAM

Quantization is the first lever: Phi-4 14B weights are 8.4 GB at Q4_K_M but 14.5 GB at Q8_0, so the same model costs nearly twice as much before a single token of context. The second lever is the KV-cache layout: classic GQA models fall into a size-class estimate of 128-384 KB per token, while hybrid linear-attention models such as the Qwen3.5 and Qwen3-Next rows above carry an exact per-token cost from their published config, because only their sparse full-attention layers cache KV. Context length is the third lever and it multiplies that per-token cost linearly, which is why the KV column in this table is charged separately from the weights. Add all three and a 27B from one family can need a different card than a 27B from another.

FAQ

Frequently asked questions

How much VRAM does a 7B model need?

Weights at Q4_K_M cost about 0.6 GB per billion parameters, so a 7B model comes in near 4.2 GB. In the probed set behind this page, Mistral 7B is 4.1 GB and DeepSeek-R1-Distill 7B is 4.4 GB at Q4_K_M, with the Q8_0 mirrors at 7.2 and 7.5 GB. An 8 GB card offers roughly 7 GB of usable budget after the 90% overhead rule, so a 7B at Q4 fits comfortably, and the Q8 build fits only with a short context. That is why the 7B-9B class rows in the tier table land on an 8 GB card.

How much VRAM does a 70B model need?

A dense 70B at Q4_K_M needs about 42 GB of weights by the 0.6 GB per billion rule, and the KV cache is charged separately: 10 GB more at 32k context on the GQA size-class estimate. Nothing below the 96 GB tier clears 52 GB of total load - the 32 GB tier only has about 29 GB usable - which is why a 70B-class build starts at the 96 GB tier (about 86 GB usable). The closest probed row on this page, Qwen3-Next 80B-A3B (a mixture-of-experts model with 3B active), loads 45.2 GB at Q4_K_M and needs the same tier.

Does Q8 need twice the VRAM of Q4?

Not exactly, and the probed files say so: across the seven size-class rows, the Q8_0 file runs from 1.3x (Gemma 3 1B Instruct) to 1.7x (Qwen3-Next 80B-A3B) of the Q4_K_M size, so "twice" overstates the typical case. Q4_K_M and Q8_0 are both partial quantizations with different group sizes and headroom, and the gap tends to widen as models grow. The safe planning range for a Q8 mirror build is about 1.3x to 1.7x the Q4 size, and the KV cache is unchanged, so Q8 also leaves less room for context.

Why do two 8B models from different families differ in VRAM?

A size class is a family of files, not one file. Qwen3 8B is 4.7 GB at Q4_K_M while Llama 3.1 8B is 4.6 GB - both 8B, both Q4, different bytes, and the same at Q8_0 (8.1 vs 8 GB). The difference comes from vocab and embedding size, layer count and KV-head layout, and small architectural extras baked into the GGUF. Context widens the gap further because the KV cache is not proportional to parameters either: a 9B with a classic layout estimates 128 KB per token in the size-class model, while the hybrid linear-attention Qwen3.5 9B caches 32 KB per token from its published config.

Related tools

Go deeper

More surfaces: all model pages · can I run it · best hardware for a model · how much VRAM for LLMs · open dataset · hardware stats