Updated 2026-10-31· Reference card: RTX 5080

Best LLM for 16GB VRAM

Short answer: The best local LLM for 16 GB of VRAM is Gemma 4 12B: 8 GB at Q4_K_M, est. 96 tok/s (typical error 15%) on the RTX 5080, the reference 16 GB card. 16 GB leaves ~14 GB for weights, enough for dense models up to ~14B at Q4; 70 of 147 local models fit (ModelFit, updated 2026-10-31).

Usable for weights
~14 GB
Largest dense model
~14B
Models that fit
70 of 147
Cards with 16 GB
11
Table 1Ranked on the RTX 5080

Best local LLMs for 16 GB of VRAM

10 models

The engine's top 8 models that fit fully, then the best 2 that only run with part of the model in system RAM. One row per model; other quant builds are listed under the name.

ModelParamsQuantVRAM (GB)Fit in 16 GBSpeed on RTX 5080
Gemma 4 12Balso Q8_012BQ4_K_M8 GBFits fullyest. 96 tok/stypical error 15%
Qwen3.5 9B Instruct (Q8)also Q4_K_M9BQ8_010.7 GBFits fullyest. 91 tok/stypical error 5%
Mellum2.1 12B-A2.5B Thinking12.1B2.5B activeQ4_K_M8.1 GBFits fullyest. 235 tok/stypical error 15%
Gemma 4 E4B (Q8)also Q4_K_M4.5BQ8_07.5 GBFits fullyest. 141 tok/stypical error 5%
NVIDIA Nemotron 3.5 Bolt E4B4BQ4_K_M7.9 GBFits fullyest. 172 tok/stypical error 5%
Ministral 3 14B13.5BQ4_K_M9.1 GBFits fullyest. 84 tok/stypical error 15%
LFM2 24B-A2B Instruct24B2B activeQ4_K_M14 GBFits fullyest. 255 tok/stypical error 15%
Ornith 1.0 9B9BQ4_K_M5.6 GBFits fullyest. 140 tok/stypical error 5%
Qwen3.5 35B-A3B Instructalso Q8_035B3B activeQ4_K_M20 GBWith offloadest. 181 tok/srough estimate, offload not measured
Gemma 4 26B-A4Balso Q8_026B4B activeQ4_K_M16 GBWith offloadest. 125 tok/srough estimate, offload not measured

VRAM is the loaded size of the build (weights plus runtime overhead) against ~14 GB usable (90% of 16 GB). Speeds are ModelFit estimates on the RTX 5080 (960 GB/s), not measured runs. Updated 2026-10-31.

Table 2By card

Every 16 GB card

11 cards

Each card's own pick, as its page shows it, and the speed of Gemma 4 12B on it. The same models fit every card of this size; bandwidth and GPU generation set the speed.

CardBandwidthIts best pickPick speedGemma 4 12B
RTX 4060 Ti288 GB/sMellum2.1 12B-A2.5B Thinkingest. 86 tok/stypical error 15%est. 30 tok/stypical error 15%
RX 9060 XT320 GB/sMellum2.1 12B-A2.5B Thinkingest. 83 tok/stypical error 25%est. 32 tok/stypical error 10%
RTX 5060 Ti448 GB/sGemma 4 12Best. 48 tok/stypical error 15%est. 48 tok/stypical error 15%
Tesla T4300 GB/sMellum2.1 12B-A2.5B Thinkingest. 75 tok/stypical error 15%est. 26 tok/stypical error 15%
Tesla P100732 GB/sMellum2.1 12B-A2.5B Thinkingest. 84 tok/stypical error 15%est. 30 tok/stypical error 15%
RX 9070 XT640 GB/sGemma 4 12Best. 58 tok/stypical error 10%est. 58 tok/stypical error 10%
RTX 4070 Ti SUPER672 GB/sGemma 4 12Best. 67 tok/stypical error 15%est. 67 tok/stypical error 15%
RTX 5070 Ti896 GB/sGemma 4 12Best. 90 tok/stypical error 15%est. 90 tok/stypical error 15%
RTX 4080716.8 GB/sGemma 4 12Best. 71 tok/stypical error 15%est. 71 tok/stypical error 15%
RTX 4080 SUPER736 GB/sGemma 4 12Best. 73 tok/stypical error 15%est. 73 tok/stypical error 15%
RTX 5080reference card960 GB/sGemma 4 12Best. 96 tok/stypical error 15%est. 96 tok/stypical error 15%

How these numbers are made

  • Picks and order: the ModelFit engine scores each of the 147 local models on the RTX 5080 for fit in 90% of VRAM, quality, speed, popularity and release date. It is the engine behind the calculator, the VRAM calculator and every GPU page.
  • Sizes: the loaded size of each build from the open dataset (/api/dataset/, CC BY 4.0, browsable on /data/). Context adds a KV cache on top; the VRAM per model size page prices it.
  • Speed: decode tokens per second at 4k of context from the fitted bandwidth model, each with its typical error (the median gap to measured runs on hardware the model was not fitted on). A row with CPU offload has no measured band. How the estimates work.
FAQ

Frequently asked questions

How big a model fits in 16 GB of VRAM?

ModelFit budgets 90% of the card for the model, about 14 GB of a 16 GB card. On the RTX 5080 the engine rates dense models up to ~14B at Q4 as a comfortable fit, and mixture-of-experts models up to ~24B total parameters. The KV cache for long context comes out of the same budget.

Can 16 GB of VRAM run a 32B model?

Not fully. A 32B model at Q4 does not fit in 16 GB, so part of it sits in system RAM and the card waits on PCIe: the RTX 5080 drops to est. 8 tok/s (rough estimate, offload not measured). A lower quant, a smaller model or a card with more VRAM keeps it on the GPU.

Which 16 GB card is fastest for local LLMs?

On the same model, speed follows memory bandwidth and the GPU generation. Gemma 4 12B runs at est. 96 tok/s (typical error 15%) on the RTX 5080 (960 GB/s), the fastest of the 11 cards with 16 GB that ModelFit tracks, and at est. 26 tok/s (typical error 15%) on the Tesla T4 (300 GB/s). All of them fit the same models.

Other sizes

Best LLM for other VRAM sizes

More: best LLMs for a 16 GB Mac · GPU VRAM calculator · VRAM for a 70B model · RAM vs VRAM for LLMs