Updated 2026-10-31· 147 local models

Best LLM by VRAM

Short answer: ModelFit's engine picks Ornith 1.0 9B for 8 GB of VRAM, Mellum2.1 12B-A2.5B Thinking for 12 GB, Gemma 4 12B for 16 GB, and Qwen3.8 27B for 24 GB, all Q4_K_M builds using 5.6 to 16.5 GB, each sized to 90% of the card and timed on one reference card per size (ModelFit, updated 2026-10-31).

Table 1By VRAM size

The best local LLM for each VRAM size

4 sizes

One row per size, timed on the calculator's reference card for that size. Open a size for the full ranking and every card of that size.

VRAMBest pickQuantVRAM usedSpeedLargest denseModels that fit
8 GB~7 GB usableOrnith 1.0 9B9B paramsQ4_K_M5.6 GBest. 43 tok/stypical error 5%on the RTX 4060~9B46 / 147
12 GB~11 GB usableMellum2.1 12B-A2.5B Thinking12.1B paramsQ4_K_M8.1 GBest. 98 tok/stypical error 15%on the RTX 3060~14B60 / 147
16 GB~14 GB usableGemma 4 12B12B paramsQ4_K_M8 GBest. 96 tok/stypical error 15%on the RTX 5080~14B70 / 147
24 GB~22 GB usableQwen3.8 27B27B paramsQ4_K_M16.5 GBest. 51 tok/stypical error 5%on the RTX 4090~32B99 / 147

Usable = 90% of the card. Speeds are ModelFit estimates, not measured runs. Updated 2026-10-31.

How these numbers are made

  • Picks: the ModelFit engine ranks each of the 147 local models on one reference card per size for fit in 90% of VRAM, quality, speed, popularity and release date, the same engine as the calculator, the VRAM calculator and every GPU page.
  • Sizes: loaded size per build from the open dataset (/api/dataset/, CC BY 4.0). The KV cache for long context comes on top.
  • Speed: decode tokens per second at 4k of context from the fitted bandwidth model, each with its typical error (the median gap to measured runs on hardware the model was not fitted on). A row with CPU offload has no measured band. How the estimates work.
Related

Size the model instead