Ornith 1.0 9B
Local OKOKB · 22
Best for: Agentic coding on small machines·Pop: 76/100
Perf: est. 58 tok/s, typical error 5% · first token ~0.5s
Fits in 8 GB VRAM with room to spare. Best for agentic coding on small machines on RTX 2070.
Best Local AI Models for RTX 2070 (8GB)
The RTX 2070 has 8GB of GDDR6 VRAM at 448 GB/s, the same capacity and bandwidth as the newer RTX 3060 Ti and RTX 5060. It holds one model up to 9B at Q4, and an 8B model runs at est. 54 tok/s, typical error 5%. Ollama still supports it: its NVIDIA table lists the RTX 2070 under compute capability 7.5.
Quick answer
The best local LLM for the RTX 2070 is Ornith 1.0 9B (est. 58 tok/s, typical error 5%) on its 8GB VRAM. It uses ~5.6GB of VRAM; the RTX 2070 handles dense models up to ~9B at Q4. A 14B model at Q4 with CPU offload runs at est. 6 tok/s, rough estimate, offload not measured. (ModelFit, 2026)
Sizing rule: a Q4 model needs about 0.6 GB of VRAM per billion parameters, and ModelFit budgets 90% of this card's 8GB for weights, context, and KV-cache. The per-size table below uses that same budget. Other strong fits: Qwen3.5 4B Instruct (Q8) (4B, ~4.3GB) and LFM2.5 8B-A1B (8.3B, ~5.5GB). On this card Ornith 1.0 9B runs at est. 58 tok/s, typical error 5%. What it will not run: a 14B model does not fit in VRAM alone, so it spills to system RAM over PCIe and slows sharply.
$ollama run ornith:9b
- Top pick
- Ornith 1.0 9B
- Decode speed
- est. 58 tok/s, typical error 5%
- VRAM needed
- ~5.6 GB
Speeds are ModelFit estimates from memory bandwidth and model size, not measured benchmarks. Typical error: the median gap between the ModelFit speed model and measured runs on hardware the model was not fitted on, for this backend and model size.
Cite this page: ModelFit, RTX 2070 VRAM: 8GB for Local LLMs (2026), 8B at est. 54 tok/s, https://modelfit.io/gpu/rtx-2070/, updated October 2026, CC BY 4.0.
Specs
VRAM
8 GB GDDR6
Speed (8B Q4)
est. 54 tok/stypical error 5%
Bandwidth
448 GB/s
Architecture
Turing
Max model size
Up to ~9B dense
Compatibility
10 excellent, 0 workable
Vendor
Table 1
RTX 2070 estimated tokens/sec by model size
Q4_K_M · ModelFit estimate| Model size | Est. speed | Fit on 8GB |
|---|---|---|
| 7B | est. 62 tok/stypical error 5% | Fits in VRAM |
| 14B | est. 6 tok/srough estimate, offload not measured | CPU offload (slow) |
| 20B MoE (3.6B active) | est. 25 tok/srough estimate, offload not measured | CPU offload (slow) |
| 32B | est. 1 tok/srough estimate, offload not measured | CPU offload (slow) |
| 35B MoE (3B active) | est. 30 tok/srough estimate, offload not measured | CPU offload (slow) |
| 70B | est. 1 tok/srough estimate, offload not measured | CPU offload (slow) |
| 120B MoE (5.1B active) | est. 18 tok/srough estimate, offload not measured | CPU offload (slow) |
Fig. 1RTX 2070: estimated speed by model size
Decode speed, generic Q4_K_M model of each size at 4K context, 448 GB/s, typical error 5%
Fits in 8 GBCPU offload (slow)
7B62
14B6
20B MoE (3.6B active)25
32B1
35B MoE (3B active)30
70B1
120B MoE (5.1B active)18
tok/s est.ModelFit estimates, not measured benchmarks: decode speed of a generic Q4_K_M model of each size at 4K context on the RTX 2070's 448 GB/s bandwidth. MoE rows scale by active parameters (decode reads only the active experts), so a 35B MoE runs far faster than a dense 32B. "CPU offload" sizes exceed the 8GB VRAM; dense models slow to a crawl there, MoE models degrade less because hot experts stay GPU-resident. Offload rows are rough estimates: no measured run covers offload, so they carry no typical error. What typical error means.
Fig. 2
Context costs VRAM too.
Ornith 1.0 9B loads ~5.6 GB of weights; at 16k context the KV cache adds ~2.0 GB (exceeds the ~7 GB usable VRAM), and at 64k it adds ~8.0 GB (exceeds the budget, use a smaller quant or a q8_0 KV cache).
Weights 5.6 GBKV cache · 16k context 2.0 GBRuntime reserve 0.8 GB
This configuration exceeds the comfortable VRAM budget by about 0.4 GB. Expect offload pressure or a shorter safe context.
KV-cache figures assume an fp16 cache, the llama.cpp/Ollama default. Standard GQA models use a size-class estimate (8 KV heads x 128 head dim class); hybrid linear-attention models (Qwen3.5/3.6, Qwen3-Next) use the exact per-token cost from their published config, since only their sparse full-attention layers cache KV. A q8_0 KV cache roughly halves either figure. Estimates, not measurements.
Get told when a model beats Ornith 1.0 9B on your RTX 2070
One email when a new open-weight model is a better fit for 8 GB of VRAM, plus the Thursday weekly.
You'll also get the weekly ModelFit email. By signing up you agree to our Privacy Policy. Unsubscribe from either email anytime, in one click.
Where to buy the RTX 2070
Storage & accessories for your model library
- Internal NVMe SSD · 2TBA Gen4 M.2 drive keeps your whole GGUF and quant collection on fast local storage, loading models straight off NVMe.Check price on Amazon
- USB4 NVMe Enclosure40Gbps external storage fast enough to run models from. Pair it with an M.2 drive for a portable model vault.Check price on Amazon
ModelFit may earn a commission on purchases through these links, at no extra cost to you.
RTX 2070 VRAM for AI: what actually fits?
ModelFit budgets about 7.2GB of the 8GB for weights and context. Qwen3 8B at Q4 takes ~6.5GB, leaving little room for the KV cache; Qwen 2.5 7B and Mistral 7B (~5.5GB each) leave more. A 14B model spills into system RAM, where the estimate falls to est. 6 tok/s, rough estimate, offload not measured. Decode speed follows memory bandwidth, so the 2070 runs 8B models a little behind the RTX 3060 Ti (est. 61 tok/s, typical error 5%) and ahead of the RTX 4060 (est. 40 tok/s, 272 GB/s). An upgrade for local LLMs means more memory: a 12GB or 16GB card changes what you can load.
Table 2
What does not fit in 8GB (and what it costs you)
With 8GB, the honest ceiling is a single 7B model at Q4. Anything larger does not simply run slower: the layers that do not fit spill to system RAM over PCIe, and decode speed falls off a cliff. These are the same bandwidth-derived estimates used in the table above, shown here as the penalty rather than the headline.
| Model size | Fits in 8GB? | Est. speed | Slowdown vs 7B |
|---|---|---|---|
| 7B | Yes | est. 62 tok/stypical error 5% | n/a |
| 14B | No, needs ~2GB more in system RAM | est. 6 tok/srough estimate, offload not measured | ~10x slower |
| 20B MoE (3.6B active) | No, needs ~5GB more in system RAM | est. 25 tok/srough estimate, offload not measured | ~2x slower |
| 32B | No, needs ~12GB more in system RAM | est. 1 tok/srough estimate, offload not measured | ~62x slower |
| 35B MoE (3B active) | No, needs ~14GB more in system RAM | est. 30 tok/srough estimate, offload not measured | ~2x slower |
Mixture-of-experts models are the exception worth knowing: they activate only a fraction of their parameters per token, so a large MoE can stay usable on a small card where a dense model of the same total size will not. Speeds are ModelFit estimates from memory bandwidth and model size, not measured benchmarks.
Table 3
RTX 2070 vs similar GPUs
| Hardware | Memory | Speed | Bandwidth |
|---|---|---|---|
| RTX 4060 | 8 GB | est. 40 tok/stypical error 5% | 272 GB/s |
| RTX 2070 | 8 GB | est. 54 tok/stypical error 5% | 448 GB/s |
| RTX 3060 Ti | 8 GB | est. 61 tok/stypical error 5% | 448 GB/s |
| RTX 3070 | 8 GB | est. 61 tok/stypical error 5% | 448 GB/s |
| RTX 3060 | 12 GB | est. 50 tok/stypical error 5% | 360 GB/s |
Recommended models
10 modelsQwen3.5 4B Instruct (Q8)
Local OKExcellentA · 40
Best for: Coding, Agents, Multimodal·Pop: 88/100
Perf: est. 63 tok/s, typical error 5% · first token ~1.5s
Fits in 8 GB VRAM with room to spare. Best for coding, agents, multimodal on RTX 2070.
LFM2.5 8B-A1B
Local OKOKB · 24
Best for: On-device agents, tool calling, multilingual chat·Pop: 72/100
Perf: est. 153 tok/s, typical error 15% · first token ~0.1s
Fits in 8 GB VRAM with room to spare. Best for on-device agents, tool calling, multilingual chat on RTX 2070.
Gemma 4 E4B
Local OKExcellentA · 44
Best for: On-device, Mobile, Chat·Pop: 82/100
Perf: est. 81 tok/s, typical error 5% · first token ~0.5s
Fits in 8 GB VRAM with room to spare. Best for on-device, mobile, chat on RTX 2070.
MiMo-V2.6-Distill-Qwen-9B
Local OKOKB · 19
Best for: Agentic coding, Small machines·Pop: 60/100
Perf: est. 56 tok/s, typical error 5% · first token ~0.5s
Fits in 8 GB VRAM with room to spare. Best for agentic coding, small machines on RTX 2070.
Gemma 4 E2B (Q8)
Local OKExcellentA · 36
Best for: IoT, Mobile, Edge·Pop: 76/100
Perf: est. 127 tok/s, typical error 5% · first token ~1.6s
Fits in 8 GB VRAM with room to spare. Best for iot, mobile, edge on RTX 2070.
Granite 4.2 8B
Local OKExcellentA · 31
Best for: Chat, Coding·Pop: 66/100
Perf: est. 51 tok/s, typical error 5% · first token ~0.6s
Fits in 8 GB VRAM with room to spare. Best for chat, coding on RTX 2070.
Qwen3.5 9B Instruct
Local OKOKB · 3
Best for: Quality, Coding, Reasoning·Pop: 86/100
Perf: est. 58 tok/s, typical error 5% · first token ~0.5s
Fits in 8 GB VRAM with room to spare. Best for quality, coding, reasoning on RTX 2070.
Ministral 3 8B
Local OKOKB · 17
Best for: Chat, Coding, Vision·Pop: 74/100
Perf: est. 48 tok/s, typical error 5% · first token ~0.6s
Fits in 8 GB VRAM with room to spare. Best for chat, coding, vision on RTX 2070.
Granite Switch 4.1 8B
Local OKOKB · 11
Best for: Chat, Decisions, Safety checks·Pop: 64/100
Perf: est. 42 tok/s, typical error 5% · first token ~0.6s
Fits in 8 GB VRAM with room to spare. Best for chat, decisions, safety checks on RTX 2070.
Video and image models on the RTX 2070
Runs · Stated: the linked source says a GPU with that much VRAM runs the model. Runs · Published: a dated run on the named card with that much VRAM. unknown: no source yet. Fits with offload: the main model weights fit in VRAM with a 15% margin; the text encoder and VAE run from system RAM, which is slower. Exceeds VRAM: even the main weights are too large for 8 GB. Every VRAM tier: video · image. Ranked with reasons: video models for 8 GB of VRAM · image models for 8 GB of VRAM.
Video
| Model | Fits | Quality | App |
|---|---|---|---|
| LTX-2.3LTX-2 Community License Agreement | Runs · PublishedRTX 4060 · 8 GB VRAM · ComfyUI GGUF workflow, 720p 5 s in about 190 s, 32 GB RAM · ComfyUI issue #14345 | No independent score yet | Lightricks/LTX-2 ltx-pipelines, ComfyUI |
| MiniMax H3MiniMax H3 Community License Agreement | Runs · PublishedRTX 4050 · 6 GB VRAM · ComfyUI int8 convrot, 0.4 megapixel, 5 s, 16 GB RAM · ComfyUI issue #15482 | #8 of 48 in text to video11,417 votessnapshot 2026-09-22 | stable-diffusion.cpp, ComfyUI |
| Wan2.2 A14BApache License 2.0 | Runs · PublishedRTX 2060 · 6 GB VRAM · ComfyUI Q8 GGUF, 480x720, 81 frames, 7 to 10 min, 28 GB RAM plus a 60 GB page file · ComfyUI issue #12451 | #42 of 48 in text to video10,458 votessnapshot 2026-09-22 | ComfyUI |
| Wan2.2 TI2V 5BApache License 2.0 | Runs · Stated8 GB VRAM · ComfyUI native offloading · ComfyUI docs: Wan2.2Complete: Comfy-Org repackaged FP16 with FP8 scaled UMT518.1 GB download | No independent score yet | ComfyUI |
| Mochi 1 previewApache License 2.0 | unknownComplete: calcuis GGUF Q3_K_M with Q4_0 T5 and FP8 VAE7.7 GB download | #48 of 48 in text to video5,893 votessnapshot 2026-09-22 | Diffusers, ComfyUI |
Quality: blind human votes from Arena (arena.ai), CC BY 4.0, snapshot 2026-09-22. Scores are for the Arena checkpoint, not your local quantized build.
Image
| Model | Fits | Quality | App |
|---|---|---|---|
| FLUX 3 Image KleinFLUX Kommunity License v1.0 | Runs · Stated8 GB VRAM · official ComfyUI int8 files · FLUX 3 Image Klein README (Black Forest Labs) | No independent score yet | Diffusers |
| FLUX.1 [dev]FLUX.1 [dev] Non-Commercial License v1.1.1 | Runs · Stated6 GB VRAM · stable-diffusion.cpp with GGUF weights · stable-diffusion.cpp FLUX docs | #78 of 82 in text to image49,221 votessnapshot 2026-10-07 | Diffusers, ComfyUI |
| Qwen-Image-2.1Qwen Research License Agreement | Runs · Stated7 GB VRAM · DiffSynth-Studio, weights streamed from disk · DiffSynth-Studio docs: Qwen-Image-2.1 | #19 of 82 in text to image11,441 votessnapshot 2026-10-07 | Diffusers, stable-diffusion.cpp |
| SDXL Base 1.0CreativeML Open RAIL++-M License | Runs · Stated8 GB VRAM · SDXL 1.0, no app named · SDXL 1.0 announcement (Stability AI) | No independent score yet | ComfyUI, stable-diffusion.cpp |
| Z-Image-TurboApache 2.0 | Runs · Stated4 GB VRAM · stable-diffusion.cpp with GGUF weights · stable-diffusion.cpp Z-Image docs | #59 of 82 in text to image24,808 votessnapshot 2026-10-07 | Diffusers, stable-diffusion.cpp |
| FLUX.2 [klein]Apache 2.0 (4B), FLUX Non-Commercial License (9B) | Fits with offloadComplete: Klein 4B Q8_0 GGUF12.5 GB download | No independent score yet | Diffusers, stable-diffusion.cpp |
| MOSS-ImageApache 2.0 | Fits with offloadComplete: UD-Q4_K_XL GGUF13.2 GB download | No independent score yet | stable-diffusion.cpp |
| Stable Diffusion 3.5 LargeStability AI Community License | Fits with offloadComplete: Q4_0 GGUF16.4 GB download | #81 of 82 in text to image23,396 votessnapshot 2026-10-07 | ComfyUI |
| FLUX.2 [dev]FLUX Non-Commercial License | Exceeds VRAMComplete: Q4_K_M GGUF34.8 GB download | #38 of 82 in text to image77,988 votessnapshot 2026-10-07 | Diffusers, ComfyUI, stable-diffusion.cpp |
| Krea 2 TurboKrea 2 Community License Agreement v.1 | Exceeds VRAMComplete: Turbo Q4_K_M GGUF16.9 GB download | No independent score yet | Diffusers, stable-diffusion.cpp |
Quality: blind human votes from Arena (arena.ai), CC BY 4.0, snapshot 2026-10-07. Scores are for the Arena checkpoint, not your local quantized build.
Similar GPUs for local AI
- RTX 3060 Ti8GB · est. 61 tok/s
- RTX 306012GB · est. 50 tok/s
- RTX 40608GB · est. 40 tok/s
- RTX 30708GB · est. 61 tok/s
Speeds are estimates for Qwen3 8B Q4_K_M, typical error 5%.
Compatible model families
- QwenAlibaba Cloud: Qwen3.8 27B local flagship (Apache 2.0, vision)
- LlamaMeta: Most popular open-weight model family
- DeepSeekDeepSeek AI: V4.1 Flash: multimodal 552B MoE, API only
- MistralMistral AI: Excellent performance-per-parameter ratio
- GemmaGoogle DeepMind: Gemma 4 E4B: 4.5B effective, runs on 8GB Macs
- PhiMicrosoft: Best quality-per-gigabyte at small sizes
RTX 2070 FAQ: common questions
How much VRAM does the RTX 2070 have?
The RTX 2070 has 8GB of GDDR6 VRAM on a 256-bit bus at 448 GB/s (NVIDIA Turing whitepaper). ModelFit budgets about 7.2GB of it for weights and context, enough for one model up to 9B at Q4 quantization.
What size LLM can I run on an RTX 2070?
Up to 9B at Q4 quantization: Qwen3 8B (~6.5GB), Llama 3.1 8B (~6.5GB) or Mistral 7B (~5.5GB). A 14B model spills into system RAM, where the estimate falls to est. 6 tok/s, rough estimate, offload not measured.
Does Ollama still support the RTX 2070?
Yes. Ollama lists the RTX 2070 under compute capability 7.5 in its NVIDIA support table and needs driver version 550 or newer. It detects the card through CUDA with no extra setup.
RTX 2070 vs RTX 3060 for local AI?
The RTX 3060 has 12GB against 8GB, so it runs 14B models at Q4 that the 2070 cannot hold. The 2070 streams memory faster (448 vs 360 GB/s), so on 8B models it runs est. 54 tok/s, typical error 5% against est. 50 tok/s. For more model choice, the 3060 12GB wins.
How fast is a 27B-class model on the RTX 2070?
The RTX 2070's 8GB of VRAM cannot fit a 32B model comfortably. The largest size class it fits is 7B, at est. 62 tok/s, typical error 5%. The largest individual model in the catalog that fits is GLM-4-9B-Chat (9B).
Related guides & benchmarks
- Best AI Models for 8GB VRAM (2026)Ranked 3B-8B picks for 8GB cards like the RTX 2070.
- How Much VRAM Do You Need for Local LLMs?A tier-by-tier table from 8GB to 96GB: what each size of card can run.
- Gemma 4 GPU Requirements: Every VariantWhich Gemma 4 variants fit an 8GB GPU, and which need 12GB or more.
More local AI guides
- Best LLM for 8GB VRAMEvery 8 GB card side by side, each with its best pick and speed.
- How Much RAM (or VRAM) Do You Need for a Local LLM?The model-size-to-memory matrix: what each VRAM and RAM tier actually runs.
- Best LLM for MacBook (Apple Silicon)Unified-memory Macs run bigger models per dollar. Picks by RAM tier, M1 to M5.
- Local AI image generator on MacCompare image models by RAM and app.
- Local image generation on MacImage models and apps by Mac RAM.
Browse all NVIDIA GPUs for AI
Real-world calibration
Have this hardware? Share a measured run.
Time a model that fits your machine with Ollama, using the same prompts as every other run. The CLI shows your result and asks before it sends anything; your machine then joins the speed test leaderboard. It does not change the quality benchmark.
$ npx @modelfit/cli@latest bench
Want personalized recommendations?
Use our interactive wizard to compare models across Apple Silicon and NVIDIA GPUs.