Updated 2026-10-31· Reference card: RTX 4090

Best LLM for 24GB VRAM

Short answer: The best local LLM for 24 GB of VRAM is Qwen3.8 27B: 16.5 GB at Q4_K_M, est. 51 tok/s (typical error 5%) on the RTX 4090, the reference 24 GB card. 24 GB leaves ~22 GB for weights, enough for dense models up to ~32B at Q4; 99 of 147 local models fit (ModelFit, updated 2026-10-31).

Usable for weights
~22 GB
Largest dense model
~32B
Models that fit
99 of 147
Cards with 24 GB
4
Table 1Ranked on the RTX 4090

Best local LLMs for 24 GB of VRAM

10 models

The engine's top 8 models that fit fully, then the best 2 that only run with part of the model in system RAM. One row per model; other quant builds are listed under the name.

ModelParamsQuantVRAM (GB)Fit in 24 GBSpeed on RTX 4090
Qwen3.8 27Balso Q8_027BQ4_K_M16.5 GBFits fullyest. 51 tok/stypical error 5%
Gemma 4 26B-A4Balso Q8_026B4B activeQ4_K_M16 GBFits fullyest. 168 tok/stypical error 15%
Gemma 4 12B (Q8)also Q4_K_M12BQ8_012.8 GBFits fullyest. 61 tok/stypical error 15%
Nemotron 3.5 Lightning 30B-A3B (IQ4_XS)also Q4_K_M30B3B activeIQ4_XS18.9 GBFits fullyest. 200 tok/stypical error 15%
Qwen3.5 35B-A3B Instructalso Q8_035B3B activeQ4_K_M20 GBFits fullyest. 242 tok/stypical error 15%
LFM2 24B-A2B Instruct24B2B activeQ4_K_M14 GBFits fullyest. 257 tok/stypical error 15%
Gemma 4 31Balso Q8_031BQ4_K_M20 GBFits fullyest. 41 tok/stypical error 5%
AREX-2 27B27.4BQ4_K_M17.2 GBFits fullyest. 50 tok/stypical error 5%
Qwen3.6 35B-A3Balso Q8_035B3B activeQ4_K_M22 GBWith offloadest. 179 tok/srough estimate, offload not measured
Sage-A1.535.95B3B activeQ4_K_M21.7 GBWith offloadest. 179 tok/srough estimate, offload not measured

VRAM is the loaded size of the build (weights plus runtime overhead) against ~22 GB usable (90% of 24 GB). Speeds are ModelFit estimates on the RTX 4090 (1008 GB/s), not measured runs. Updated 2026-10-31.

Table 2By card

Every 24 GB card

4 cards

Each card's own pick, as its page shows it, and the speed of Qwen3.8 27B on it. The same models fit every card of this size; bandwidth and GPU generation set the speed.

CardBandwidthIts best pickPick speedQwen3.8 27B
RX 7900 XTX960 GB/sQwen3.8 27Best. 44 tok/stypical error 50%est. 44 tok/stypical error 50%
RTX 3090936 GB/sQwen3.8 27Best. 45 tok/stypical error 5%est. 45 tok/stypical error 5%
RTX 3090 Ti1008 GB/sQwen3.8 27Best. 48 tok/stypical error 5%est. 48 tok/stypical error 5%
RTX 4090reference card1008 GB/sQwen3.8 27Best. 51 tok/stypical error 5%est. 51 tok/stypical error 5%

How these numbers are made

  • Picks and order: the ModelFit engine scores each of the 147 local models on the RTX 4090 for fit in 90% of VRAM, quality, speed, popularity and release date. It is the engine behind the calculator, the VRAM calculator and every GPU page.
  • Sizes: the loaded size of each build from the open dataset (/api/dataset/, CC BY 4.0, browsable on /data/). Context adds a KV cache on top; the VRAM per model size page prices it.
  • Speed: decode tokens per second at 4k of context from the fitted bandwidth model, each with its typical error (the median gap to measured runs on hardware the model was not fitted on). A row with CPU offload has no measured band. How the estimates work.
FAQ

Frequently asked questions

How big a model fits in 24 GB of VRAM?

ModelFit budgets 90% of the card for the model, about 22 GB of a 24 GB card. On the RTX 4090 the engine rates dense models up to ~32B at Q4 as a comfortable fit, and mixture-of-experts models up to ~35B total parameters. The KV cache for long context comes out of the same budget.

Can 24 GB of VRAM run a 70B model?

Not fully. A 70B model at Q4 does not fit in 24 GB, so part of it sits in system RAM and the card waits on PCIe: the RTX 4090 drops to est. 2 tok/s (rough estimate, offload not measured). A lower quant, a smaller model or a card with more VRAM keeps it on the GPU.

Which 24 GB card is fastest for local LLMs?

On the same model, speed follows memory bandwidth and the GPU generation. Qwen3.8 27B runs at est. 51 tok/s (typical error 5%) on the RTX 4090 (1008 GB/s), the fastest of the 4 cards with 24 GB that ModelFit tracks, and at est. 44 tok/s (typical error 50%) on the RX 7900 XTX (960 GB/s). All of them fit the same models.

Other sizes

Best LLM for other VRAM sizes

More: Best AI Models for 24GB VRAM (2026) · best LLMs for a 24 GB Mac · GPU VRAM calculator · VRAM for a 70B model · RAM vs VRAM for LLMs