Updated 2026-10-31· Reference card: RTX 3060

Best LLM for 12GB VRAM

Short answer: The best local LLM for 12 GB of VRAM is Mellum2.1 12B-A2.5B Thinking: 8.1 GB at Q4_K_M, est. 98 tok/s (typical error 15%) on the RTX 3060, the reference 12 GB card. 12 GB leaves ~11 GB for weights, enough for dense models up to ~14B at Q4; 60 of 147 local models fit (ModelFit, updated 2026-10-31).

Usable for weights
~11 GB
Largest dense model
~14B
Models that fit
60 of 147
Cards with 12 GB
5
Table 1Ranked on the RTX 3060

Best local LLMs for 12 GB of VRAM

10 models

The engine's top 8 models that fit fully, then the best 2 that only run with part of the model in system RAM. One row per model; other quant builds are listed under the name.

ModelParamsQuantVRAM (GB)Fit in 12 GBSpeed on RTX 3060
Mellum2.1 12B-A2.5B Thinking12.1B2.5B activeQ4_K_M8.1 GBFits fullyest. 98 tok/stypical error 15%
Qwen3.5 9B Instructalso Q8_09BQ4_K_M7 GBFits fullyest. 53 tok/stypical error 5%
Ornith 1.0 9B9BQ4_K_M5.6 GBFits fullyest. 53 tok/stypical error 5%
Gemma 4 12Balso Q8_012BQ4_K_M8 GBFits fullyest. 35 tok/stypical error 15%
LFM2.5 8B-A1B8.3B1.5B activeQ4_K_M5.5 GBFits fullyest. 139 tok/stypical error 15%
NVIDIA Nemotron 3.5 Bolt E4B4BQ4_K_M7.9 GBFits fullyest. 67 tok/stypical error 5%
Gemma 4 E4B (Q8)also Q4_K_M4.5BQ8_07.5 GBFits fullyest. 53 tok/stypical error 5%
MiMo-V2.6-Distill-Qwen-9B9BQ4_K_M5.8 GBFits fullyest. 51 tok/stypical error 5%
Gemma 4 26B-A4Balso Q8_026B4B activeQ4_K_M16 GBWith offloadest. 49 tok/srough estimate, offload not measured
LFM2 24B-A2B Instruct24B2B activeQ4_K_M14 GBWith offloadest. 82 tok/srough estimate, offload not measured

VRAM is the loaded size of the build (weights plus runtime overhead) against ~11 GB usable (90% of 12 GB). Speeds are ModelFit estimates on the RTX 3060 (360 GB/s), not measured runs. Updated 2026-10-31.

Table 2By card

Every 12 GB card

5 cards

Each card's own pick, as its page shows it, and the speed of Mellum2.1 12B-A2.5B Thinking on it. The same models fit every card of this size; bandwidth and GPU generation set the speed.

CardBandwidthIts best pickPick speedMellum2.1 12B-A2.5B Thinking
RTX 3060reference card360 GB/sMellum2.1 12B-A2.5B Thinkingest. 98 tok/stypical error 15%est. 98 tok/stypical error 15%
RTX 4070504 GB/sQwen3.5 9B Instructest. 78 tok/stypical error 5%est. 141 tok/stypical error 15%
RTX 4070 SUPER504 GB/sQwen3.5 9B Instructest. 78 tok/stypical error 5%est. 141 tok/stypical error 15%
RTX 4070 Ti504 GB/sQwen3.5 9B Instructest. 78 tok/stypical error 5%est. 141 tok/stypical error 15%
RTX 5070672 GB/sGemma 4 12Best. 70 tok/stypical error 15%est. 182 tok/stypical error 15%

How these numbers are made

  • Picks and order: the ModelFit engine scores each of the 147 local models on the RTX 3060 for fit in 90% of VRAM, quality, speed, popularity and release date. It is the engine behind the calculator, the VRAM calculator and every GPU page.
  • Sizes: the loaded size of each build from the open dataset (/api/dataset/, CC BY 4.0, browsable on /data/). Context adds a KV cache on top; the VRAM per model size page prices it.
  • Speed: decode tokens per second at 4k of context from the fitted bandwidth model, each with its typical error (the median gap to measured runs on hardware the model was not fitted on). A row with CPU offload has no measured band. How the estimates work.
FAQ

Frequently asked questions

How big a model fits in 12 GB of VRAM?

ModelFit budgets 90% of the card for the model, about 11 GB of a 12 GB card. On the RTX 3060 the engine rates dense models up to ~14B at Q4 as a comfortable fit. The KV cache for long context comes out of the same budget.

Can 12 GB of VRAM run a 32B model?

Not fully. A 32B model at Q4 does not fit in 12 GB, so part of it sits in system RAM and the card waits on PCIe: the RTX 3060 drops to est. 1 tok/s (rough estimate, offload not measured). A lower quant, a smaller model or a card with more VRAM keeps it on the GPU.

Which 12 GB card is fastest for local LLMs?

On the same model, speed follows memory bandwidth and the GPU generation. Mellum2.1 12B-A2.5B Thinking runs at est. 182 tok/s (typical error 15%) on the RTX 5070 (672 GB/s), the fastest of the 5 cards with 12 GB that ModelFit tracks, and at est. 98 tok/s (typical error 15%) on the RTX 3060 (360 GB/s). All of them fit the same models.

Other sizes

Best LLM for other VRAM sizes

More: GPU VRAM calculator · VRAM for a 70B model · RAM vs VRAM for LLMs