GPU VRAM Calculator
Which local LLM fits on your graphics card, card by card: usable VRAM, the largest model each card holds, and the best pick with its estimated speed.
What can my GPU run? Weights at Q4 cost about 0.6 GB per billion parameters, against a usable budget of roughly 90% of your VRAM. On the RTX 5080 (16 GB, the calculator's default tier), that is ~14 GB of weights, which the engine pairs with Gemma 4 E4B (Q8) at about ~95 tok/s (est.). Every card below is graded the same way: VRAM fixes the ceiling, bandwidth sets the speed, and the table shows what each one actually fits.
With 16 GB of VRAM, ModelFit budgets about 14 GB for the model and comfortably runs local LLMs up to ~14B parameters at Q4. The best single pick is Gemma 4 E4B (Q8). Estimates use a representative 16 GB card (RTX 5080).
Tokens/sec are ModelFit estimates from memory bandwidth and model size, not measured benchmarks. Ollama commands are registry-verified.
Every card, what it fits
One row per discrete card in ModelFit's catalog, computed by the same engine the calculator runs: the 90% weights budget, then the largest and best-scoring local models that fit. The GPU hub lists the same cards side by side.
Q4_K_M builds from ModelFit's 106-model catalog. Fit and tok/s are ModelFit estimates from memory bandwidth and model size, not measured benchmarks. Updated 2026-09-18.
Speed by model size on three popular cards
These are bandwidth-derived estimates, not measured benchmarks: each card's rate scales from its own memory bandwidth and the 90% usable budget. Rows marked CPU offload exceed the card's VRAM; MoE rows scale by active parameters, so they degrade far less when offloaded.
| Model size | Est. tok/s | Fits? |
|---|---|---|
| 7B | ~34 tok/s | Fits in VRAM |
| 14B | ~4 tok/s | CPU offload (slow) |
| 20B MoE (3.6B active) (MoE) | ~10 tok/s | CPU offload (slow) |
| 32B | ~1 tok/s | CPU offload (slow) |
| 35B MoE (3B active) (MoE) | ~9 tok/s | CPU offload (slow) |
| 70B | not practical | CPU offload (slow) |
| 120B MoE (5.1B active) (MoE) | ~4 tok/s | CPU offload (slow) |
| Model size | Est. tok/s | Fits? |
|---|---|---|
| 7B | ~117 tok/s | Fits in VRAM |
| 14B | ~65 tok/s | Fits in VRAM |
| 20B MoE (3.6B active) (MoE) | ~99 tok/s | Fits in VRAM |
| 32B | ~32 tok/s | Fits in VRAM |
| 35B MoE (3B active) (MoE) | ~84 tok/s | Fits in VRAM |
| 70B | ~2 tok/s | CPU offload (slow) |
| 120B MoE (5.1B active) (MoE) | ~14 tok/s | CPU offload (slow) |
| Model size | Est. tok/s | Fits? |
|---|---|---|
| 7B | ~162 tok/s | Fits in VRAM |
| 14B | ~90 tok/s | Fits in VRAM |
| 20B MoE (3.6B active) (MoE) | ~138 tok/s | Fits in VRAM |
| 32B | ~45 tok/s | Fits in VRAM |
| 35B MoE (3B active) (MoE) | ~117 tok/s | Fits in VRAM |
| 70B | ~23 tok/s | Fits in VRAM |
| 120B MoE (5.1B active) (MoE) | ~19 tok/s | CPU offload (slow) |
Anchored to an 8B-class Q4_K_M model and scaled by model size; estimates, not measured benchmarks. Updated 2026-09-18.
How to read your card's spec sheet
A graphics card spec sheet lists dozens of numbers, but two decide everything about local LLMs. VRAM is the hard ceiling: at Q4 the weights cost about 0.6 GB per billion parameters, and ModelFit budgets 90% of the card for them, which is why the table above rounds every card to its real usable budget. Memory bandwidth sets the speed once the model fits, because wider memory moves tokens out faster, and every tok/s figure on this page is an estimate scaled from it. Clock speeds and core counts move performance by a few percent; these two numbers decide whether the model runs at all and how it feels while it does.
Frequently asked questions
Is 8GB of VRAM enough for local LLMs in 2026?
Eight gigabytes still covers the small-model tier. The 90% usable budget leaves about 7 GB for weights, which holds an 8B model at Q4 (5 GB) with modest context and powers lightweight chat and coding assistants; the RTX 4060 row of the table above shows exactly what the engine pairs with it. A 14B model needs about 8 GB of weights, so an 8GB card only reaches it with heavier quantization or CPU offload, both of which cost quality and speed. If 14B-class is your floor, a 12GB or 16GB card is the more comfortable starting point.
Does memory bandwidth matter as much as VRAM?
They answer different questions. VRAM decides what fits: nothing runs if the weights do not fit the budget. Bandwidth decides how fast it runs once it does, because decode speed is roughly proportional to it for a given model size, which is why every tok/s figure on this page is bandwidth-derived. The practical consequence is that a faster older card with less VRAM can beat a roomier one on small models, so the best-pick column weighs fit first and speed second. Mixture-of-experts models bend the rule in your favor: only their active experts are read per token, so decode speed tracks active parameters, not the total.
Which GPU do I need for a 32B model?
A dense 32B at Q4 needs about 19 GB of weights, and against the 90% budget that implies a card with at least 22 GB of VRAM. In practice that means a 24GB card, such as the RTX 3090, RTX 4090 or RX 7900 XTX, which all hold it comfortably per the table above. A 16GB card lands at about 14 GB usable, roughly 5 GB short of a dense 32B, so it falls back to lower quantizations or CPU offload.
Can I pool two GPUs to run a bigger model?
Yes, and that is exactly what the multi-GPU rig mode of the calculator below computes. Two 16GB cards pool about 29 GB of usable weight budget, which clears the 19 GB a dense 32B needs. The honest caveat is that pooled throughput is lower than a single card with the same total VRAM, because layers are transferred between cards, and mixture-of-experts models tolerate pooling better since only their active experts cross the link. Pooling is the practical path to 70B-class models on consumer cards, which is why the rig mode exists.
Go deeper
By VRAM amount instead of by card, with the KV-cache math for your context length.
Weights plus the context cache plus the budget your machine actually hands over.
The question in reverse: start from a model size and see which cards hold it.
Why unified memory and dedicated video memory answer the same question differently.
Unified memory, single card and multi-GPU rigs in one tool, including the pooled fit this page points to.
Every card ranked by estimated tokens per second per dollar, refreshed quarterly.
More surfaces: all GPU pages · can I run it · best hardware for a model · quantization comparison · open compatibility dataset · hardware stats
The weekly local-AI refresh
New open-weight models, real Apple Silicon benchmarks, and the one model worth running on your Mac this week. Free, one email a week, unsubscribe anytime.
By subscribing you agree to our Privacy Policy and to receive the weekly email. Unsubscribe anytime.