Updated 2026-10-31· Reference card: RTX 4060

Best LLM for 8GB VRAM

Short answer: The best local LLM for 8 GB of VRAM is Ornith 1.0 9B: 5.6 GB at Q4_K_M, est. 43 tok/s (typical error 5%) on the RTX 4060, the reference 8 GB card. 8 GB leaves ~7 GB for weights, enough for dense models up to ~9B at Q4; 46 of 147 local models fit (ModelFit, updated 2026-10-31).

Usable for weights
~7 GB
Largest dense model
~9B
Models that fit
46 of 147
Cards with 8 GB
5
Table 1Ranked on the RTX 4060

Best local LLMs for 8 GB of VRAM

10 models

The engine's top 8 models that fit fully, then the best 2 that only run with part of the model in system RAM. One row per model; other quant builds are listed under the name.

ModelParamsQuantVRAM (GB)Fit in 8 GBSpeed on RTX 4060
Ornith 1.0 9B9BQ4_K_M5.6 GBFits fullyest. 43 tok/stypical error 5%
LFM2.5 8B-A1B8.3B1.5B activeQ4_K_M5.5 GBFits fullyest. 118 tok/stypical error 15%
Qwen3.5 4B Instruct (Q8)also Q4_K_M4BQ8_04.3 GBFits fullyest. 47 tok/stypical error 5%
Gemma 4 E4Balso Q8_04.5BQ4_K_M4 GBFits fullyest. 61 tok/stypical error 5%
MiMo-V2.6-Distill-Qwen-9B9BQ4_K_M5.8 GBFits fullyest. 42 tok/stypical error 5%
Gemma 4 E2B (Q8)also Q4_K_M2.3BQ8_04.6 GBFits fullyest. 97 tok/stypical error 5%
Qwen3.5 9B Instructalso Q8_09BQ4_K_M7 GBFits fullyest. 43 tok/stypical error 5%
Granite 4.2 8B8.8BQ4_K_M5 GBFits fullyest. 38 tok/stypical error 5%
Mellum2.1 12B-A2.5B Thinking12.1B2.5B activeQ4_K_M8.1 GBWith offloadest. 61 tok/srough estimate, offload not measured
NVIDIA Nemotron 3.5 Bolt E4B4BQ4_K_M7.9 GBWith offloadest. 11 tok/srough estimate, offload not measured

VRAM is the loaded size of the build (weights plus runtime overhead) against ~7 GB usable (90% of 8 GB). Speeds are ModelFit estimates on the RTX 4060 (272 GB/s), not measured runs. Updated 2026-10-31.

Table 2By card

Every 8 GB card

5 cards

Each card's own pick, as its page shows it, and the speed of Ornith 1.0 9B on it. The same models fit every card of this size; bandwidth and GPU generation set the speed.

CardBandwidthIts best pickPick speedOrnith 1.0 9B
RTX 4060reference card272 GB/sOrnith 1.0 9Best. 43 tok/stypical error 5%est. 43 tok/stypical error 5%
RTX 2070448 GB/sOrnith 1.0 9Best. 58 tok/stypical error 5%est. 58 tok/stypical error 5%
RTX 3070448 GB/sOrnith 1.0 9Best. 65 tok/stypical error 5%est. 65 tok/stypical error 5%
RTX 3060 Ti448 GB/sOrnith 1.0 9Best. 65 tok/stypical error 5%est. 65 tok/stypical error 5%
RTX 5060448 GB/sOrnith 1.0 9Best. 73 tok/stypical error 5%est. 73 tok/stypical error 5%

How these numbers are made

  • Picks and order: the ModelFit engine scores each of the 147 local models on the RTX 4060 for fit in 90% of VRAM, quality, speed, popularity and release date. It is the engine behind the calculator, the VRAM calculator and every GPU page.
  • Sizes: the loaded size of each build from the open dataset (/api/dataset/, CC BY 4.0, browsable on /data/). Context adds a KV cache on top; the VRAM per model size page prices it.
  • Speed: decode tokens per second at 4k of context from the fitted bandwidth model, each with its typical error (the median gap to measured runs on hardware the model was not fitted on). A row with CPU offload has no measured band. How the estimates work.
FAQ

Frequently asked questions

How big a model fits in 8 GB of VRAM?

ModelFit budgets 90% of the card for the model, about 7 GB of an 8 GB card. On the RTX 4060 the engine rates dense models up to ~9B at Q4 as a comfortable fit. The KV cache for long context comes out of the same budget.

Can 8 GB of VRAM run a 14B model?

Not fully. A 14B model at Q4 does not fit in 8 GB, so part of it sits in system RAM and the card waits on PCIe: the RTX 4060 drops to est. 5 tok/s (rough estimate, offload not measured). A lower quant, a smaller model or a card with more VRAM keeps it on the GPU.

Which 8 GB card is fastest for local LLMs?

On the same model, speed follows memory bandwidth and the GPU generation. Ornith 1.0 9B runs at est. 73 tok/s (typical error 5%) on the RTX 5060 (448 GB/s), the fastest of the 5 cards with 8 GB that ModelFit tracks, and at est. 43 tok/s (typical error 5%) on the RTX 4060 (272 GB/s). All of them fit the same models.

Other sizes

Best LLM for other VRAM sizes

More: Best AI Models for 8GB VRAM (2026): 6 Local LLMs Ranked · best LLMs for a 8 GB Mac · GPU VRAM calculator · VRAM for a 70B model · RAM vs VRAM for LLMs