Qwen3.5 9B Instruct
Local OKOKB · 22
Best for: Quality, Coding, Reasoning·Pop: 86/100
Perf: est. 105 tok/s, typical error 5% · first token ~0.3s
Fits in 10 GB VRAM with room to spare. Best for quality, coding, reasoning on RTX 3080 10GB.
Best Local AI Models for RTX 3080 (10GB)
The RTX 3080 10GB pairs 10GB of GDDR6X with a 320-bit bus at 760 GB/s. Token generation follows memory bandwidth, so an 8B model runs at est. 99 tok/s, typical error 5%, ahead of the RTX 3070 (est. 61 tok/s) and the RTX 3060 (est. 50 tok/s). The limit is capacity: 10GB holds dense models up to 12B at Q4.
Quick answer
The best local LLM for the RTX 3080 10GB is Qwen3.5 9B Instruct (est. 105 tok/s, typical error 5%) on its 10GB VRAM. It uses ~7GB of VRAM; the RTX 3080 10GB handles dense models up to ~12B at Q4. A 14B model at Q4 with CPU offload runs at est. 12 tok/s, rough estimate, offload not measured. (ModelFit, 2026)
Sizing rule: a Q4 model needs about 0.6 GB of VRAM per billion parameters, and ModelFit budgets 90% of this card's 10GB for weights, context, and KV-cache. The per-size table below uses that same budget. Other strong fits: Ornith 1.0 9B (9B, ~5.6GB) and Gemma 4 12B (12B, ~8GB). On this card Qwen3.5 9B Instruct runs at est. 105 tok/s, typical error 5%. What it will not run: a 14B model does not fit in VRAM alone, so it spills to system RAM over PCIe and slows sharply.
$ollama run qwen3.5:9b
- Top pick
- Qwen3.5 9B Instruct
- Decode speed
- est. 105 tok/s, typical error 5%
- VRAM needed
- ~7 GB
Speeds are ModelFit estimates from memory bandwidth and model size, not measured benchmarks. Typical error: the median gap between the ModelFit speed model and measured runs on hardware the model was not fitted on, for this backend and model size.
Cite this page: ModelFit, RTX 3080 10GB for Local LLMs (2026): Up to 12B, 8B at est. 99 tok/s, https://modelfit.io/gpu/rtx-3080-10gb/, updated November 2026, CC BY 4.0.
Specs
VRAM
10 GB GDDR6X
Speed (8B Q4)
est. 99 tok/stypical error 5%
Bandwidth
760 GB/s
Architecture
Ampere
Max model size
Up to ~12B dense
Compatibility
10 excellent, 0 workable
Table 1
RTX 3080 10GB estimated tokens/sec by model size
Q4_K_M · ModelFit estimate| Model size | Est. speed | Fit on 10GB |
|---|---|---|
| 7B | est. 112 tok/stypical error 5% | Fits in VRAM |
| 14B | est. 12 tok/srough estimate, offload not measured | CPU offload (slow) |
| 20B MoE (3.6B active) | est. 96 tok/srough estimate, offload not measured | CPU offload (slow) |
| 32B | est. 3 tok/srough estimate, offload not measured | CPU offload (slow) |
| 35B MoE (3B active) | est. 52 tok/srough estimate, offload not measured | CPU offload (slow) |
| 70B | est. 1 tok/srough estimate, offload not measured | CPU offload (slow) |
| 120B MoE (5.1B active) | est. 34 tok/srough estimate, offload not measured | CPU offload (slow) |
Fig. 1RTX 3080 10GB: estimated speed by model size
Decode speed, generic Q4_K_M model of each size at 4K context, 760 GB/s, typical error 5%
Fits in 10 GBCPU offload (slow)
7B112
14B12
20B MoE (3.6B active)96
32B3
35B MoE (3B active)52
70B1
120B MoE (5.1B active)34
tok/s est.ModelFit estimates, not measured benchmarks: decode speed of a generic Q4_K_M model of each size at 4K context on the RTX 3080 10GB's 760 GB/s bandwidth. MoE rows scale by active parameters (decode reads only the active experts), so a 35B MoE runs far faster than a dense 32B. "CPU offload" sizes exceed the 10GB VRAM; dense models slow to a crawl there, MoE models degrade less because hot experts stay GPU-resident. Offload rows are rough estimates: no measured run covers offload, so they carry no typical error. What typical error means.
Fig. 2
Context costs VRAM too.
Qwen3.5 9B Instruct loads ~7 GB of weights; at 16k context the KV cache adds ~0.5 GB (still fits the ~9 GB usable VRAM), and at 64k it adds ~2.0 GB (still fits).
Weights 7.0 GBKV cache · 16k context 0.5 GBHeadroom 1.5 GBRuntime reserve 1.0 GB
The model leaves about 1.5 GB of the usable VRAM budget free after weights and 16k context.
KV-cache figures assume an fp16 cache, the llama.cpp/Ollama default. Standard GQA models use a size-class estimate (8 KV heads x 128 head dim class); hybrid linear-attention models (Qwen3.5/3.6, Qwen3-Next) use the exact per-token cost from their published config, since only their sparse full-attention layers cache KV. A q8_0 KV cache roughly halves either figure. Estimates, not measurements.
Get told when a model beats Qwen3.5 9B Instruct on your RTX 3080 10GB
One email when a new open-weight model is a better fit for 10 GB of VRAM, plus the Thursday weekly.
You'll also get the weekly ModelFit email. By signing up you agree to our Privacy Policy. Unsubscribe from either email anytime, in one click.
Where to buy the RTX 3080 10GB
New cards with 10 GB of VRAM or more
exact products- RTX 3060 · 12 GB VRAM12 GB GDDR6 at 360 GB/s · Qwen3 8B Q4 est. 50 tok/sView on Amazon
- RTX 4070 · 12 GB VRAM12 GB GDDR6X at 504 GB/s · Qwen3 8B Q4 est. 73 tok/sView on Amazon
- RTX 4070 SUPER · 12 GB VRAM12 GB GDDR6X at 504 GB/s · Qwen3 8B Q4 est. 73 tok/sView on Amazon
ModelFit may earn a commission on purchases through these links, at no extra cost to you.
Storage & accessories for your model library
- Internal NVMe SSD · 2TBA Gen4 M.2 drive keeps your whole GGUF and quant collection on fast local storage, loading models straight off NVMe.Check price on Amazon
- USB4 NVMe Enclosure40Gbps external storage fast enough to run models from. Pair it with an M.2 drive for a portable model vault.Check price on Amazon
ModelFit may earn a commission on purchases through these links, at no extra cost to you.
RTX 3080 10GB VRAM for AI: what actually fits?
ModelFit budgets about 9GB of the 10GB for weights and context. Qwen3 8B at Q4 takes ~6.5GB, which leaves room for the KV cache; Qwen 2.5 7B and Mistral 7B (~5.5GB each) leave more. The RTX 3060 has 2GB more VRAM but less than half the bandwidth (360 GB/s), so the 3080 is the faster card on every model both can hold.
Table 2
RTX 3080 10GB vs similar GPUs
Recommended models
10 modelsOrnith 1.0 9B
Local OKExcellentA · 38
Best for: Agentic coding on small machines·Pop: 76/100
Perf: est. 105 tok/s, typical error 5% · first token ~0.3s
Fits in 10 GB VRAM with room to spare. Best for agentic coding on small machines on RTX 3080 10GB.
Gemma 4 12B
Local OKOKB · 11
Best for: Chat, Coding, Multimodal·Pop: 80/100
Perf: est. 71 tok/s, typical error 15% · first token ~0.4s
Fits in 10 GB VRAM with room to spare. Best for chat, coding, multimodal on RTX 3080 10GB.
Gemma 4 E4B (Q8)
Local OKOKB · 17
Best for: On-device, Mobile, Chat·Pop: 82/100
Perf: est. 105 tok/s, typical error 5% · first token ~1.5s
Fits in 10 GB VRAM with room to spare. Best for on-device, mobile, chat on RTX 3080 10GB.
LFM2.5 8B-A1B
Local OKExcellentA · 39
Best for: On-device agents, tool calling, multilingual chat·Pop: 72/100
Perf: est. 238 tok/s, typical error 15% · first token ~0.1s
Fits in 10 GB VRAM with room to spare. Best for on-device agents, tool calling, multilingual chat on RTX 3080 10GB.
MiMo-V2.6-Distill-Qwen-9B
Local OKExcellentA · 36
Best for: Agentic coding, Small machines·Pop: 60/100
Perf: est. 102 tok/s, typical error 5% · first token ~0.3s
Fits in 10 GB VRAM with room to spare. Best for agentic coding, small machines on RTX 3080 10GB.
Mellum2.1 12B-A2.5B Thinking
Local OKOKB · 10
Best for: Agentic coding, Small machines·Pop: 76/100
Perf: est. 179 tok/s, typical error 15% · first token ~0.1s
Fits in 10 GB VRAM with room to spare. Best for agentic coding, small machines on RTX 3080 10GB.
NVIDIA Nemotron 3.5 Bolt E4B
Local OKOKB · 12
Best for: Voice assistants, Tool calling, Chat·Pop: 75/100
Perf: est. 130 tok/s, typical error 5% · first token ~0.3s
Fits in 10 GB VRAM with room to spare. Best for voice assistants, tool calling, chat on RTX 3080 10GB.
Granite Switch 4.1 8B
Local OKOKB · 29
Best for: Chat, Decisions, Safety checks·Pop: 64/100
Perf: est. 78 tok/s, typical error 5% · first token ~0.3s
Fits in 10 GB VRAM with room to spare. Best for chat, decisions, safety checks on RTX 3080 10GB.
Granite 4.2 8B
Local OKExcellentA · 44
Best for: Chat, Coding·Pop: 66/100
Perf: est. 94 tok/s, typical error 5% · first token ~0.3s
Fits in 10 GB VRAM with room to spare. Best for chat, coding on RTX 3080 10GB.
Video and image models on the RTX 3080 10GB
Runs · Stated: the linked source says a GPU with that much VRAM runs the model. Runs · Published: a dated run on the named card with that much VRAM. unknown: no source yet. Fits with offload: the main model weights fit in VRAM with a 15% margin; the text encoder and VAE run from system RAM, which is slower. Exceeds VRAM: even the main weights are too large for 10 GB. Every VRAM tier: video · image. Ranked with reasons: video models for 8 GB of VRAM · image models for 8 GB of VRAM.
Video
| Model | Fits | Quality | App |
|---|---|---|---|
| LTX-2.3LTX-2 Community License Agreement | Runs · PublishedRTX 4060 · 8 GB VRAM · ComfyUI GGUF workflow, 720p 5 s in about 190 s, 32 GB RAM · ComfyUI issue #14345 | No independent score yet | Lightricks/LTX-2 ltx-pipelines, ComfyUI |
| MiniMax H3MiniMax H3 Community License Agreement | Runs · PublishedRTX 4050 · 6 GB VRAM · ComfyUI int8 convrot, 0.4 megapixel, 5 s, 16 GB RAM · ComfyUI issue #15482 | #8 of 48 in text to video11,417 votessnapshot 2026-09-22 | stable-diffusion.cpp, ComfyUI |
| Wan2.2 A14BApache License 2.0 | Runs · PublishedRTX 2060 · 6 GB VRAM · ComfyUI Q8 GGUF, 480x720, 81 frames, 7 to 10 min, 28 GB RAM plus a 60 GB page file · ComfyUI issue #12451 | #42 of 48 in text to video10,458 votessnapshot 2026-09-22 | ComfyUI |
| Wan2.2 TI2V 5BApache License 2.0 | Runs · Stated8 GB VRAM · ComfyUI native offloading · ComfyUI docs: Wan2.2Complete: Comfy-Org repackaged FP16 with FP8 scaled UMT518.1 GB download | No independent score yet | ComfyUI |
| Mochi 1 previewApache License 2.0 | unknownComplete: calcuis GGUF Q3_K_M with Q4_0 T5 and FP8 VAE7.7 GB download | #48 of 48 in text to video5,893 votessnapshot 2026-09-22 | Diffusers, ComfyUI |
Quality: blind human votes from Arena (arena.ai), CC BY 4.0, snapshot 2026-09-22. Scores are for the Arena checkpoint, not your local quantized build.
Image
| Model | Fits | Quality | App |
|---|---|---|---|
| FLUX 3 Image KleinFLUX Kommunity License v1.0 | Runs · Stated8 GB VRAM · official ComfyUI int8 files · FLUX 3 Image Klein README (Black Forest Labs) | No independent score yet | Diffusers |
| FLUX.1 [dev]FLUX.1 [dev] Non-Commercial License v1.1.1 | Runs · Stated6 GB VRAM · stable-diffusion.cpp with GGUF weights · stable-diffusion.cpp FLUX docs | #78 of 82 in text to image49,221 votessnapshot 2026-10-07 | Diffusers, ComfyUI |
| Qwen-Image-2.1Qwen Research License Agreement | Runs · Stated7 GB VRAM · DiffSynth-Studio, weights streamed from disk · DiffSynth-Studio docs: Qwen-Image-2.1 | #19 of 82 in text to image11,441 votessnapshot 2026-10-07 | Diffusers, stable-diffusion.cpp |
| SDXL Base 1.0CreativeML Open RAIL++-M License | Runs · Stated8 GB VRAM · SDXL 1.0, no app named · SDXL 1.0 announcement (Stability AI) | No independent score yet | ComfyUI, stable-diffusion.cpp |
| Z-Image-TurboApache 2.0 | Runs · Stated4 GB VRAM · stable-diffusion.cpp with GGUF weights · stable-diffusion.cpp Z-Image docs | #59 of 82 in text to image24,808 votessnapshot 2026-10-07 | Diffusers, stable-diffusion.cpp |
| FLUX.2 [klein]Apache 2.0 (4B), FLUX Non-Commercial License (9B) | Fits with offloadComplete: Klein 4B Q8_0 GGUF12.5 GB download | No independent score yet | Diffusers, stable-diffusion.cpp |
| Krea 2 TurboKrea 2 Community License Agreement v.1 | Fits with offloadComplete: Turbo Q4_K_M GGUF16.9 GB download | No independent score yet | Diffusers, stable-diffusion.cpp |
| MOSS-ImageApache 2.0 | Fits with offloadComplete: UD-Q4_K_XL GGUF13.2 GB download | No independent score yet | stable-diffusion.cpp |
| Stable Diffusion 3.5 LargeStability AI Community License | Fits with offloadComplete: Q4_0 GGUF16.4 GB download | #81 of 82 in text to image23,396 votessnapshot 2026-10-07 | ComfyUI |
| FLUX.2 [dev]FLUX Non-Commercial License | Exceeds VRAMComplete: Q4_K_M GGUF34.8 GB download | #38 of 82 in text to image77,988 votessnapshot 2026-10-07 | Diffusers, ComfyUI, stable-diffusion.cpp |
Quality: blind human votes from Arena (arena.ai), CC BY 4.0, snapshot 2026-10-07. Scores are for the Arena checkpoint, not your local quantized build.
Config reports
No speed test run for this GPU yet. Run the speed test: npx @modelfit/cli@latest bench
Share your config
Similar GPUs for local AI
- RTX 306012GB · est. 50 tok/s
- RTX 30708GB · est. 61 tok/s
- RTX 309024GB · est. 117 tok/s
- RTX 407012GB · est. 73 tok/s
Speeds are estimates for Qwen3 8B Q4_K_M, typical error 5%.
Compatible model families
- QwenAlibaba Cloud: Qwen3.8 27B local flagship (Apache 2.0, vision)
- LlamaMeta: Most popular open-weight model family
- DeepSeekDeepSeek AI: V4.1 Flash: multimodal 552B MoE, API only
- MistralMistral AI: Excellent performance-per-parameter ratio
- GemmaGoogle DeepMind: Gemma 4 E4B: 4.5B effective, runs on 8GB Macs
- PhiMicrosoft: Best quality-per-gigabyte at small sizes
RTX 3080 10GB FAQ: common questions
How much VRAM does the RTX 3080 10GB have?
The original RTX 3080 has 10GB of GDDR6X on a 320-bit bus at 760 GB/s (NVIDIA RTX Blackwell whitepaper, Table 4). NVIDIA also lists a 12GB RTX 3080 on a 384-bit bus. ModelFit budgets about 9GB of the 10GB for weights and context, enough for dense models up to 12B at Q4.
What size LLM can I run on an RTX 3080 10GB?
Dense models up to 12B at Q4 quantization. Qwen3 8B (~6.5GB) runs at est. 99 tok/s, typical error 5%, and Llama 3.1 8B (~6.5GB) and Mistral 7B (~5.5GB) fit with more room for context.
RTX 3080 10GB vs RTX 3060 12GB for local LLMs?
The 3060 has 2GB more VRAM; the 3080 streams 760 against 360 GB/s, so on 8B models it runs est. 99 tok/s, typical error 5% against est. 50 tok/s, about 98% faster. For the largest models that fit 12GB but not 10GB, the 3060 holds them in VRAM.
How fast is a 27B-class model on the RTX 3080 10GB?
The RTX 3080 10GB's 10GB of VRAM cannot fit a 32B model comfortably. The largest size class it fits is 7B, at est. 112 tok/s, typical error 5%. The largest individual model in the catalog that fits is Mellum2.1 12B-A2.5B Thinking (12B).
Related guides & benchmarks
More local AI guides
- How Much RAM (or VRAM) Do You Need for a Local LLM?The model-size-to-memory matrix: what each VRAM and RAM tier actually runs.
- Best LLM for MacBook (Apple Silicon)Unified-memory Macs run bigger models per dollar. Picks by RAM tier, M1 to M5.
- Local AI image generator on MacCompare image models by RAM and app.
- Local image generation on MacImage models and apps by Mac RAM.
Browse all NVIDIA GPUs for AI
Real-world calibration
Have this hardware? Share a measured run.
Time a model that fits your machine with Ollama, using the same prompts as every other run. The CLI shows your result and asks before it sends anything; your machine then joins the speed test leaderboard. It does not change the quality benchmark.
$ npx @modelfit/cli@latest bench
Want personalized recommendations?
Use our interactive wizard to compare models across Apple Silicon and NVIDIA GPUs.