Can I run it?
Exact model x GPU verdicts: fit, memory math with KV-cache, estimated speed, best quant and upgrade path. Estimates, not measurements.
RTX 3060
12 GBQwen3 8B Q4_K_MRunsQwen3 14B Q4_K_MSlowGPT-OSS 20B MXFP4SlowQwen3.8 27B Q4_K_MNoLlama 3.3 70B Instruct Q4_K_MNoQwen3.5 35B-A3B Instruct Q4_K_MNoGemma 4 26B-A4B Q4_K_MSlowQwen3.5 4B Instruct Q4_K_MRunsGemma 4 E4B Q4_K_MRunsGemma 3 4B Instruct Q4_K_MRunsLlama 3.1 8B Instruct Q4_K_MRunsQwen3.5 9B Instruct Q4_K_MRunsGemma 4 12B Q4_K_MRunsMistral Nemo 12B Q4_K_MRunsQwen3.5 9B Instruct (Q8) Q8_0RunsGemma 4 12B (Q8) Q8_0SlowLFM2 24B-A2B Instruct Q4_K_MSlowQwen3.5 27B Instruct Q4_K_MSlowQwen3.6 27B Q4_K_MNoGemma 4 31B Q4_K_MNoQwen3.6 35B-A3B Q4_K_MNoQwen3.8 27B (Q8) Q8_0No
RTX 4060
8 GBQwen3 8B Q4_K_MRunsQwen3 14B Q4_K_MNoGPT-OSS 20B MXFP4NoQwen3.8 27B Q4_K_MNoLlama 3.3 70B Instruct Q4_K_MNoQwen3.5 35B-A3B Instruct Q4_K_MNoGemma 4 26B-A4B Q4_K_MNoQwen3.5 4B Instruct Q4_K_MRunsGemma 4 E4B Q4_K_MRunsGemma 3 4B Instruct Q4_K_MRunsLlama 3.1 8B Instruct Q4_K_MRunsQwen3.5 9B Instruct Q4_K_MRunsGemma 4 12B Q4_K_MSlowMistral Nemo 12B Q4_K_MSlowQwen3.5 9B Instruct (Q8) Q8_0SlowGemma 4 12B (Q8) Q8_0NoLFM2 24B-A2B Instruct Q4_K_MNoQwen3.5 27B Instruct Q4_K_MNoQwen3.6 27B Q4_K_MNoGemma 4 31B Q4_K_MNoQwen3.6 35B-A3B Q4_K_MNoQwen3.8 27B (Q8) Q8_0No
RTX 4060 Ti
16 GBQwen3 8B Q4_K_MRunsQwen3 14B Q4_K_MRunsGPT-OSS 20B MXFP4RunsQwen3.8 27B Q4_K_MSlowLlama 3.3 70B Instruct Q4_K_MNoQwen3.5 35B-A3B Instruct Q4_K_MSlowGemma 4 26B-A4B Q4_K_MSlowQwen3.5 4B Instruct Q4_K_MRunsGemma 4 E4B Q4_K_MRunsGemma 3 4B Instruct Q4_K_MRunsLlama 3.1 8B Instruct Q4_K_MRunsQwen3.5 9B Instruct Q4_K_MRunsGemma 4 12B Q4_K_MRunsMistral Nemo 12B Q4_K_MRunsQwen3.5 9B Instruct (Q8) Q8_0RunsGemma 4 12B (Q8) Q8_0RunsLFM2 24B-A2B Instruct Q4_K_MRunsQwen3.5 27B Instruct Q4_K_MSlowQwen3.6 27B Q4_K_MSlowGemma 4 31B Q4_K_MSlowQwen3.6 35B-A3B Q4_K_MNoQwen3.8 27B (Q8) Q8_0No
RTX 4070
12 GBQwen3 8B Q4_K_MRunsQwen3 14B Q4_K_MSlowGPT-OSS 20B MXFP4SlowQwen3.8 27B Q4_K_MNoLlama 3.3 70B Instruct Q4_K_MNoQwen3.5 35B-A3B Instruct Q4_K_MNoGemma 4 26B-A4B Q4_K_MSlowQwen3.5 4B Instruct Q4_K_MRunsGemma 4 E4B Q4_K_MRunsGemma 3 4B Instruct Q4_K_MRunsLlama 3.1 8B Instruct Q4_K_MRunsQwen3.5 9B Instruct Q4_K_MRunsGemma 4 12B Q4_K_MRunsMistral Nemo 12B Q4_K_MRunsQwen3.5 9B Instruct (Q8) Q8_0RunsGemma 4 12B (Q8) Q8_0SlowLFM2 24B-A2B Instruct Q4_K_MSlowQwen3.5 27B Instruct Q4_K_MSlowQwen3.6 27B Q4_K_MNoGemma 4 31B Q4_K_MNoQwen3.6 35B-A3B Q4_K_MNoQwen3.8 27B (Q8) Q8_0No
RTX 4070 Ti SUPER
16 GBQwen3 8B Q4_K_MRunsQwen3 14B Q4_K_MRunsGPT-OSS 20B MXFP4RunsQwen3.8 27B Q4_K_MSlowLlama 3.3 70B Instruct Q4_K_MNoQwen3.5 35B-A3B Instruct Q4_K_MSlowGemma 4 26B-A4B Q4_K_MSlowQwen3.5 4B Instruct Q4_K_MRunsGemma 4 E4B Q4_K_MRunsGemma 3 4B Instruct Q4_K_MRunsLlama 3.1 8B Instruct Q4_K_MRunsQwen3.5 9B Instruct Q4_K_MRunsGemma 4 12B Q4_K_MRunsMistral Nemo 12B Q4_K_MRunsQwen3.5 9B Instruct (Q8) Q8_0RunsGemma 4 12B (Q8) Q8_0RunsLFM2 24B-A2B Instruct Q4_K_MRunsQwen3.5 27B Instruct Q4_K_MSlowQwen3.6 27B Q4_K_MSlowGemma 4 31B Q4_K_MSlowQwen3.6 35B-A3B Q4_K_MNoQwen3.8 27B (Q8) Q8_0No
RTX 3090
24 GBQwen3 8B Q4_K_MRunsQwen3 14B Q4_K_MRunsGPT-OSS 20B MXFP4RunsQwen3.8 27B Q4_K_MRunsLlama 3.3 70B Instruct Q4_K_MNoQwen3.5 35B-A3B Instruct Q4_K_MRunsGemma 4 26B-A4B Q4_K_MRunsQwen3.5 4B Instruct Q4_K_MRunsGemma 4 E4B Q4_K_MRunsGemma 3 4B Instruct Q4_K_MRunsLlama 3.1 8B Instruct Q4_K_MRunsQwen3.5 9B Instruct Q4_K_MRunsGemma 4 12B Q4_K_MRunsMistral Nemo 12B Q4_K_MRunsQwen3.5 9B Instruct (Q8) Q8_0RunsGemma 4 12B (Q8) Q8_0RunsLFM2 24B-A2B Instruct Q4_K_MRunsQwen3.5 27B Instruct Q4_K_MRunsQwen3.6 27B Q4_K_MRunsGemma 4 31B Q4_K_MRunsQwen3.6 35B-A3B Q4_K_MSlowQwen3.8 27B (Q8) Q8_0Slow
RTX 4090
24 GBQwen3 8B Q4_K_MRunsQwen3 14B Q4_K_MRunsGPT-OSS 20B MXFP4RunsQwen3.8 27B Q4_K_MRunsLlama 3.3 70B Instruct Q4_K_MNoQwen3.5 35B-A3B Instruct Q4_K_MRunsGemma 4 26B-A4B Q4_K_MRunsQwen3.5 4B Instruct Q4_K_MRunsGemma 4 E4B Q4_K_MRunsGemma 3 4B Instruct Q4_K_MRunsLlama 3.1 8B Instruct Q4_K_MRunsQwen3.5 9B Instruct Q4_K_MRunsGemma 4 12B Q4_K_MRunsMistral Nemo 12B Q4_K_MRunsQwen3.5 9B Instruct (Q8) Q8_0RunsGemma 4 12B (Q8) Q8_0RunsLFM2 24B-A2B Instruct Q4_K_MRunsQwen3.5 27B Instruct Q4_K_MRunsQwen3.6 27B Q4_K_MRunsGemma 4 31B Q4_K_MRunsQwen3.6 35B-A3B Q4_K_MSlowQwen3.8 27B (Q8) Q8_0Slow
RTX 5070 Ti
16 GBQwen3 8B Q4_K_MRunsQwen3 14B Q4_K_MRunsGPT-OSS 20B MXFP4RunsQwen3.8 27B Q4_K_MSlowLlama 3.3 70B Instruct Q4_K_MNoQwen3.5 35B-A3B Instruct Q4_K_MSlowGemma 4 26B-A4B Q4_K_MSlowQwen3.5 4B Instruct Q4_K_MRunsGemma 4 E4B Q4_K_MRunsGemma 3 4B Instruct Q4_K_MRunsLlama 3.1 8B Instruct Q4_K_MRunsQwen3.5 9B Instruct Q4_K_MRunsGemma 4 12B Q4_K_MRunsMistral Nemo 12B Q4_K_MRunsQwen3.5 9B Instruct (Q8) Q8_0RunsGemma 4 12B (Q8) Q8_0RunsLFM2 24B-A2B Instruct Q4_K_MRunsQwen3.5 27B Instruct Q4_K_MSlowQwen3.6 27B Q4_K_MSlowGemma 4 31B Q4_K_MSlowQwen3.6 35B-A3B Q4_K_MNoQwen3.8 27B (Q8) Q8_0No
RTX 5080
16 GBQwen3 8B Q4_K_MRunsQwen3 14B Q4_K_MRunsGPT-OSS 20B MXFP4RunsQwen3.8 27B Q4_K_MSlowLlama 3.3 70B Instruct Q4_K_MNoQwen3.5 35B-A3B Instruct Q4_K_MSlowGemma 4 26B-A4B Q4_K_MSlowQwen3.5 4B Instruct Q4_K_MRunsGemma 4 E4B Q4_K_MRunsGemma 3 4B Instruct Q4_K_MRunsLlama 3.1 8B Instruct Q4_K_MRunsQwen3.5 9B Instruct Q4_K_MRunsGemma 4 12B Q4_K_MRunsMistral Nemo 12B Q4_K_MRunsQwen3.5 9B Instruct (Q8) Q8_0RunsGemma 4 12B (Q8) Q8_0RunsLFM2 24B-A2B Instruct Q4_K_MRunsQwen3.5 27B Instruct Q4_K_MSlowQwen3.6 27B Q4_K_MSlowGemma 4 31B Q4_K_MSlowQwen3.6 35B-A3B Q4_K_MNoQwen3.8 27B (Q8) Q8_0No
RTX 5090
32 GBQwen3 8B Q4_K_MRunsQwen3 14B Q4_K_MRunsGPT-OSS 20B MXFP4RunsQwen3.8 27B Q4_K_MRunsLlama 3.3 70B Instruct Q4_K_MSlowQwen3.5 35B-A3B Instruct Q4_K_MRunsGemma 4 26B-A4B Q4_K_MRunsQwen3.5 4B Instruct Q4_K_MRunsGemma 4 E4B Q4_K_MRunsGemma 3 4B Instruct Q4_K_MRunsLlama 3.1 8B Instruct Q4_K_MRunsQwen3.5 9B Instruct Q4_K_MRunsGemma 4 12B Q4_K_MRunsMistral Nemo 12B Q4_K_MRunsQwen3.5 9B Instruct (Q8) Q8_0RunsGemma 4 12B (Q8) Q8_0RunsLFM2 24B-A2B Instruct Q4_K_MRunsQwen3.5 27B Instruct Q4_K_MRunsQwen3.6 27B Q4_K_MRunsGemma 4 31B Q4_K_MRunsQwen3.6 35B-A3B Q4_K_MRunsQwen3.8 27B (Q8) Q8_0Runs