Kolibri 1
Local OKExcellentA · 42
Best for: Reasoning, German, Long context·Pop: 75/100
Perf: est. 282 tok/s, typical error 15% · first token ~0.1s
Fits in 96 GB VRAM with room to spare. Best for reasoning, german, long context on RTX PRO 6000.
Best Local AI Models for NVIDIA RTX PRO 6000 Blackwell (96GB)
The RTX PRO 6000 Blackwell is NVIDIA's workstation flagship, built for professionals rather than gamers. Its 96GB of ECC GDDR7 is three times the RTX 5090's VRAM, at the same 1,792 GB/s bandwidth. That combination lets it load 120B-class dense models entirely in memory while still generating 7B-8B model tokens as fast as the 5090.
Quick answer
The best local LLM for the RTX PRO 6000 is Kolibri 1 (est. 282 tok/s, typical error 15%) on its 96GB VRAM. It uses ~49.8GB of VRAM; the RTX PRO 6000 handles dense models up to ~123B at Q4. A 27B-class model (Qwen3.8 27B) at Q4 runs at est. 89 tok/s, typical error 5%. (ModelFit, 2026)
Sizing rule: a Q4 model needs about 0.6 GB of VRAM per billion parameters for its weights, and ModelFit budgets 90% of this card's 96GB (86.4GB) for weights, context, and KV-cache. Real files often load more than that rule, so the per-size table below checks actual builds: a size fits only when the lightest Q4 build of that size in our catalog loads within 86.4GB. MoE rows, and sizes with no such build, use a typical Q4 file size. Other strong fits: Kolibri 1 (IQ4_XS) (78.1B, ~43.6GB) and Ornith 1.5 122B (125.09B, ~75.8GB). On this card Kolibri 1 runs at est. 282 tok/s, typical error 15%. What it will not run: dense models above ~123B at Q4 exceed this budget and need a lower quant, CPU offload, or a second GPU.
- Top pick
- Kolibri 1
- Decode speed
- est. 282 tok/s, typical error 15%
- VRAM needed
- ~49.8 GB
Speeds are ModelFit estimates from memory bandwidth and model size, not measured benchmarks. Typical error: the median gap between the ModelFit speed model and measured runs on hardware the model was not fitted on, for this backend and model size.
Cite this page: ModelFit, RTX PRO 6000 Blackwell 96GB Local LLM: 120B MoE at est. 210 tok/s, https://modelfit.io/gpu/rtx-6000-pro/, updated November 2026, CC BY 4.0.
Specs
VRAM
96 GB GDDR7
Speed (8B Q4)
est. 213 tok/stypical error 5%
Bandwidth
1792 GB/s
Architecture
Blackwell
Max model size
Up to ~123B dense
Compatibility
10 excellent, 0 workable
Table 1
RTX PRO 6000 estimated tokens/sec by model size
Q4_K_M · ModelFit estimate| Model size | Est. speed | Fit on 96GB |
|---|---|---|
| 7B | est. 238 tok/stypical error 5% | Fits in VRAM |
| 14B | est. 142 tok/stypical error 15% | Fits in VRAM |
| 20B MoE (3.6B active) | est. 266 tok/stypical error 15% | Fits in VRAM |
| 32B | est. 70 tok/stypical error 5% | Fits in VRAM |
| 35B MoE (3B active) | est. 298 tok/stypical error 15% | Fits in VRAM |
| 70B | est. 34 tok/stypical error 10% | Fits in VRAM |
| 120B MoE (5.1B active) | est. 210 tok/stypical error 15% | Fits in VRAM |
Fig. 1RTX PRO 6000: estimated speed by model size
Decode speed, generic Q4_K_M model of each size at 4K context, 1792 GB/s, typical error 5-15%
7B238
14B142
20B MoE (3.6B active)266
32B70
35B MoE (3B active)298
70B34
120B MoE (5.1B active)210
tok/s est.ModelFit estimates, not measured benchmarks: decode speed of a generic Q4_K_M model of each size at 4K context on the RTX PRO 6000's 1792 GB/s bandwidth. MoE rows scale by active parameters (decode reads only the active experts), so a 35B MoE runs far faster than a dense 32B. "CPU offload" sizes exceed the 96GB VRAM; dense models slow to a crawl there, MoE models degrade less because hot experts stay GPU-resident. Offload rows are rough estimates: no measured run covers offload, so they carry no typical error. What typical error means.
Fig. 2
Context costs VRAM too.
Kolibri 1 loads ~49.8 GB of weights; at 16k context the KV cache adds ~0.3 GB (still fits the ~86 GB usable VRAM), and at 64k it adds ~1.3 GB (still fits).
Weights 50 GBKV cache · 16k context 0.3 GBHeadroom 36 GBRuntime reserve 9.6 GB
The model leaves about 36 GB of the usable VRAM budget free after weights and 16k context.
KV-cache figures assume an fp16 cache, the llama.cpp/Ollama default. Standard GQA models use a size-class estimate (8 KV heads x 128 head dim class); hybrid linear-attention models (Qwen3.5/3.6, Qwen3-Next) use the exact per-token cost from their published config, since only their sparse full-attention layers cache KV. A q8_0 KV cache roughly halves either figure. Estimates, not measurements.
Get told when a model beats Kolibri 1 on your RTX PRO 6000
One email when a new open-weight model is a better fit for 96 GB of VRAM, plus the Thursday weekly.
You'll also get the weekly ModelFit email. By signing up you agree to our Privacy Policy. Unsubscribe from either email anytime, in one click.
Where to buy the RTX PRO 6000
Storage & accessories for your model library
- Internal NVMe SSD · 2TBA Gen4 M.2 drive keeps your whole GGUF and quant collection on fast local storage, loading models straight off NVMe.Check price on Amazon
- USB4 NVMe Enclosure40Gbps external storage fast enough to run models from. Pair it with an M.2 drive for a portable model vault.Check price on Amazon
ModelFit may earn a commission on purchases through these links, at no extra cost to you.
RTX PRO 6000 VRAM for AI: what actually fits?
96GB of ECC GDDR7 at 1,792 GB/s gives the RTX PRO 6000 the same per-token speed as the RTX 5090 on models that fit both cards, since throughput is bandwidth-bound and the two cards share an identical bandwidth spec. The difference is capacity: a 70B model at Q4 (~42GB) leaves over 40GB free, and models up to roughly 120B parameters at Q4 fit inside the usable budget with room for a long context window. That headroom also means the card can hold two or three mid-size models resident at once, useful for running a chat model alongside a coding model without reloading. The 600W power draw and workstation pricing put this card well outside consumer territory; it is aimed at AI builders and studios that need the largest local models on a single GPU.
Table 2
RTX PRO 6000 vs similar GPUs
| Hardware | Memory | Speed | Bandwidth |
|---|---|---|---|
| RTX 4090 | 24 GB | est. 133 tok/stypical error 5% | 1008 GB/s |
| RTX 5090 | 32 GB | est. 213 tok/stypical error 5% | 1792 GB/s |
| RTX PRO 6000 | 96 GB | est. 213 tok/stypical error 5% | 1792 GB/s |
| Ryzen AI Max+ 395 | 110 GB | est. 32 tok/stypical error 10% | 256 GB/s |
Recommended models
10 modelsKolibri 1 (IQ4_XS)
Local OKExcellentA · 50
Best for: Reasoning, German, Long context·Pop: 75/100
Perf: est. 304 tok/s, typical error 15% · first token ~0.1s
Fits in 96 GB VRAM with room to spare. Best for reasoning, german, long context on RTX PRO 6000.
Ornith 1.5 122B
Local OKOKB · 12
Best for: Agentic coding·Pop: 74/100
Perf: est. 170 tok/s, typical error 15% · first token ~0.1s
Fits in 96 GB VRAM with room to spare. Best for agentic coding on RTX PRO 6000.
Qwen3.5 122B-A10B Instruct
Local OKOKB · 17
Best for: Frontier-level reasoning, Complex tasks·Pop: 75/100
Perf: est. 161 tok/s, typical error 15% · first token ~0.1s
Fits in 96 GB VRAM with room to spare. Best for frontier-level reasoning, complex tasks on RTX PRO 6000.
Nemotron 3 Super 120B-A12B (IQ4_XS)
Local OKOKB · 25
Best for: Agentic workflows, Long context, Reasoning·Pop: 75/100
Perf: est. 125 tok/s, typical error 15% · first token ~0.1s
Fits in 96 GB VRAM with room to spare. Best for agentic workflows, long context, reasoning on RTX PRO 6000.
Qwen3.8 27B (Q8)
Local OKExcellentA · 69
Best for: Coding, Agent, Vision, Long context·Pop: 95/100
Perf: est. 55 tok/s, typical error 5% · first token ~1.2s
Fits in 96 GB VRAM with room to spare. Best for coding, agent, vision, long context on RTX PRO 6000.
Ling-3.0-flash (IQ4_XS)
Local OKOKB · 20
Best for: Agentic coding, Long context, Reasoning·Pop: 55/100
Perf: est. 259 tok/s, typical error 15% · first token ~0.1s
Fits in 96 GB VRAM with room to spare. Best for agentic coding, long context, reasoning on RTX PRO 6000.
Qwen3.6 35B-A3B (Q8)
Local OKExcellentA · 55
Best for: Reasoning, Coding, Agents·Pop: 88/100
Perf: est. 251 tok/s, typical error 15% · first token ~0.2s
Fits in 96 GB VRAM with room to spare. Best for reasoning, coding, agents on RTX PRO 6000.
Gemma 4 26B-A4B (Q8)
Local OKExcellentA · 67
Best for: Chat, Coding, Multimodal·Pop: 86/100
Perf: est. 195 tok/s, typical error 15% · first token ~0.2s
Fits in 96 GB VRAM with room to spare. Best for chat, coding, multimodal on RTX PRO 6000.
Ling-3.0-flash
Local OKOKB · 10
Best for: Agentic coding, Long context, Reasoning·Pop: 55/100
Perf: est. 238 tok/s, typical error 15% · first token ~0.1s
Fits in 96 GB VRAM with room to spare. Best for agentic coding, long context, reasoning on RTX PRO 6000.
Video and image models on the RTX PRO 6000
Runs · Stated: the linked source says a GPU with that much VRAM runs the model. Runs · Published: a dated run on the named card with that much VRAM. unknown: no source yet. Every VRAM tier: video · image. Ranked with reasons: video models for 24 GB of VRAM · image models for 24 GB of VRAM.
Video
| Model | Fits | Quality | App |
|---|---|---|---|
| HunyuanVideo-1.5Tencent Hunyuan Community License | Runs · Stated24 GB VRAM · ComfyUI native workflow · ComfyUI docs: HunyuanVideo 1.5 | #38 of 48 in text to video4,292 votessnapshot 2026-09-22 | ComfyUI |
| LTX-2.3LTX-2 Community License Agreement | Runs · Stated24 GB VRAM · two-stage production at 720p to 1080p · LTX blog: hardware for AI video models | No independent score yet | Lightricks/LTX-2 ltx-pipelines, ComfyUI |
| LTX-2.5LTX-2.x Community License Agreement | Runs · Stated32 GB VRAM · official requirement, plus 32 GB system RAM · LTX-2.5 page (ltx.io) | No independent score yet | Lightricks/LTX-2 ltx-pipelines, ComfyUI |
| MiniMax H3MiniMax H3 Community License Agreement | Runs · Stated80 GB VRAM · Diffusers bf16 with CPU offload · Diffusers docs: MiniMax-H3 memory | #8 of 48 in text to video11,417 votessnapshot 2026-09-22 | stable-diffusion.cpp, ComfyUI |
| Mochi 1 previewApache License 2.0 | Runs · Stated42 GB VRAM · Diffusers full weights with CPU offload · Mochi 1 model card | #48 of 48 in text to video5,893 votessnapshot 2026-09-22 | Diffusers, ComfyUI |
| Wan2.1Apache License 2.0 | Runs · Stated13 GB VRAM · T2V 14B in Diffusers, bf16 with group offloading to system RAM · Diffusers docs: Wan pipeline | No independent score yet | ComfyUI |
| Wan2.2 A14BApache License 2.0 | Runs · Stated80 GB VRAM · official generate.py at 720p with offload flags · Wan2.2-T2V-A14B model card | #42 of 48 in text to video10,458 votessnapshot 2026-09-22 | ComfyUI |
| Wan2.2 TI2V 5BApache License 2.0 | Runs · Stated24 GB VRAM · official generate.py at 1280x704 with offload flags · Wan2.2-TI2V-5B model card | No independent score yet | ComfyUI |
Quality: blind human votes from Arena (arena.ai), CC BY 4.0, snapshot 2026-09-22. Scores are for the Arena checkpoint, not your local quantized build.
Image
| Model | Fits | Quality | App |
|---|---|---|---|
| FLUX 3 Image KleinFLUX Kommunity License v1.0 | Runs · Stated8 GB VRAM · official ComfyUI int8 files · FLUX 3 Image Klein README (Black Forest Labs) | No independent score yet | Diffusers |
| FLUX.1 [dev]FLUX.1 [dev] Non-Commercial License v1.1.1 | Runs · Stated10 GB VRAM · FP4 on RTX 50 (Blackwell) · Black Forest Labs blog: FLUX with NVIDIA | #78 of 82 in text to image49,221 votessnapshot 2026-10-07 | Diffusers, ComfyUI |
| FLUX.2 [dev]FLUX Non-Commercial License | Runs · Stated24 GB VRAM · 4-bit quantization in Diffusers · FLUX.2 dev Diffusers guide (Black Forest Labs) | #38 of 82 in text to image77,988 votessnapshot 2026-10-07 | Diffusers, ComfyUI, stable-diffusion.cpp |
| FLUX.2 [klein]Apache 2.0 (4B), FLUX Non-Commercial License (9B) | Runs · Stated29 GB VRAM · Klein 9B · FLUX.2 [klein] 9B model cardComplete: Klein 9B BF1634.7 GB download | No independent score yet | Diffusers, stable-diffusion.cpp |
| Krea 2 TurboKrea 2 Community License Agreement v.1 | Runs · PublishedRTX 4070 · 12 GB VRAM · ComfyUI fp8, dynamic VRAM paging, 8 steps · ComfyUI issue #15365 | No independent score yet | Diffusers, stable-diffusion.cpp |
| Qwen-Image-2.1Qwen Research License Agreement | Runs · PublishedRTX 5080 · 16 GB VRAM · stable-diffusion.cpp Q8_0, 1024x1024 · stable-diffusion.cpp issue #2041 | #19 of 82 in text to image11,441 votessnapshot 2026-10-07 | Diffusers, stable-diffusion.cpp |
| SDXL Base 1.0CreativeML Open RAIL++-M License | Runs · Stated8 GB VRAM · SDXL 1.0, no app named · SDXL 1.0 announcement (Stability AI) | No independent score yet | ComfyUI, stable-diffusion.cpp |
| Stable Diffusion 3.5 LargeStability AI Community License | Runs · Stated11 GB VRAM · FP8 TensorRT build on RTX 40 and 50 · Stability AI: SD3.5 TensorRT models | #81 of 82 in text to image23,396 votessnapshot 2026-10-07 | ComfyUI |
| Z-Image-TurboApache 2.0 | Runs · Stated16 GB VRAM · stated for consumer GPUs · Z-Image-Turbo model card | #59 of 82 in text to image24,808 votessnapshot 2026-10-07 | Diffusers, stable-diffusion.cpp |
| MOSS-ImageApache 2.0 | unknownComplete: UD-Q4_K_XL GGUF13.2 GB download | No independent score yet | stable-diffusion.cpp |
Quality: blind human votes from Arena (arena.ai), CC BY 4.0, snapshot 2026-10-07. Scores are for the Arena checkpoint, not your local quantized build.
Config reports
No speed test run for this GPU yet. Run the speed test: npx @modelfit/cli@latest bench
Share your config
Similar GPUs for local AI
Compatible model families
- QwenAlibaba Cloud: Qwen3.8 27B local flagship (Apache 2.0, vision)
- LlamaMeta: Most popular open-weight model family
- DeepSeekDeepSeek AI: V4.1 Flash: multimodal 552B MoE, API only
- MistralMistral AI: Excellent performance-per-parameter ratio
- GemmaGoogle DeepMind: Gemma 4 E4B: 4.5B effective, runs on 8GB Macs
- PhiMicrosoft: Best quality-per-gigabyte at small sizes
RTX PRO 6000 FAQ: common questions
How much VRAM does the RTX PRO 6000 Blackwell have for LLMs?
The RTX PRO 6000 Blackwell has 96GB of ECC GDDR7 VRAM with 1,792 GB/s bandwidth (NVIDIA, 2026). About 86GB is usable for model loading after driver overhead, enough for dense models up to roughly 120B parameters at Q4 quantization.
What size LLM can I run on an RTX PRO 6000 Blackwell?
Up to roughly 120B parameter models at Q4 quantization fit inside its 96GB VRAM. Smaller 7B-32B models run with a large context window and headroom to keep a second model loaded at the same time.
Is the RTX PRO 6000 Blackwell worth it for local AI?
It is a professional workstation card, not a consumer buy. Trading well above its $8,565 launch MSRP, it costs several times an RTX 5090, but it is one of the few single GPUs that holds 120B-class models entirely in VRAM without splitting across multiple cards.
RTX PRO 6000 Blackwell vs RTX 5090 for local LLMs?
Both share the same 1,792 GB/s bandwidth, so 7B-8B token speed is nearly identical (est. 213 tok/s, typical error 5%). The RTX PRO 6000 has three times the VRAM (96GB vs 32GB), letting it hold much larger models, but it costs several times more and is not built for gaming.
Does the RTX PRO 6000 Blackwell need special drivers for Ollama?
No. It uses the same NVIDIA CUDA driver stack as GeForce cards, so Ollama detects it automatically. Keep drivers current, since workstation cards often get certified driver updates on a slightly different cadence than GeForce.
How fast is a 27B-class model like Qwen3.8 27B on the RTX PRO 6000?
ModelFit estimates a 32B model on the RTX PRO 6000 at Q4_K_M runs at est. 70 tok/s, typical error 5%. The current 27B-class pick in the catalog is Qwen3.8 27B (ollama run qwen3.8:27b).
Related guides & benchmarks
More local AI guides
- How Much RAM (or VRAM) Do You Need for a Local LLM?The model-size-to-memory matrix: what each VRAM and RAM tier actually runs.
- Best LLM for MacBook (Apple Silicon)Unified-memory Macs run bigger models per dollar. Picks by RAM tier, M1 to M5.
- Local AI image generator on MacCompare image models by RAM and app.
- Local image generation on MacImage models and apps by Mac RAM.
Browse all NVIDIA GPUs for AI
Real-world calibration
Have this hardware? Share a measured run.
Time a model that fits your machine with Ollama, using the same prompts as every other run. The CLI shows your result and asks before it sends anything; your machine then joins the speed test leaderboard. It does not change the quality benchmark.
$ npx @modelfit/cli@latest bench
Want personalized recommendations?
Use our interactive wizard to compare models across Apple Silicon and NVIDIA GPUs.