The Tesla P100 PCIe 16GB is a Pascal data-center card with 16GB of HBM2 at 732 GB/s. It holds dense models up to 14B at Q4: an 8B model runs at est. 42 tok/s, typical error 5% and a 14B model at est. 25 tok/s, typical error 15%. Ollama still supports it, but compute capability 6.0 needs NVIDIA driver 570 or newer.
The best local LLM for the Tesla P100 is Mellum2.1 12B-A2.5B Thinking (est. 84 tok/s, typical error 15%) on its 16GB VRAM. It uses ~8.1GB of VRAM; the Tesla P100 handles dense models up to ~14B at Q4. A 14B model at Q4 runs at est. 25 tok/s, typical error 15%. (ModelFit, 2026)
Sizing rule: a Q4 model needs about 0.6 GB of VRAM per billion parameters, and ModelFit budgets 90% of this card's 16GB for weights, context, and KV-cache. The per-size table below uses that same budget. Other strong fits: Gemma 4 12B (12B, ~8GB) and NVIDIA Nemotron 3.5 Bolt E4B (4B, ~7.9GB). On this card Mellum2.1 12B-A2.5B Thinking runs at est. 84 tok/s, typical error 15%. What it will not run: a 32B model does not fit in VRAM alone, so it spills to system RAM over PCIe and slows sharply.
Speeds are ModelFit estimates from memory bandwidth and model size, not measured benchmarks. Typical error: the median gap between the ModelFit speed model and measured runs on hardware the model was not fitted on, for this backend and model size.
Cite this page: ModelFit, Tesla P100 16GB for Local LLMs: HBM2, 14B at est. 25 tok/s, https://modelfit.io/gpu/tesla-p100-pcie-16gb/, updated October 2026, CC BY 4.0.
| Model size | Est. speed | Fit on 16GB |
|---|---|---|
| 7B | est. 49 tok/stypical error 5% | Fits in VRAM |
| 14B | est. 25 tok/stypical error 15% | Fits in VRAM |
| 20B MoE (3.6B active) | est. 57 tok/stypical error 15% | Fits in VRAM |
| 32B | est. 2 tok/srough estimate, offload not measured | CPU offload (slow) |
| 35B MoE (3B active) | est. 23 tok/srough estimate, offload not measured | CPU offload (slow) |
| 70B | est. 1 tok/srough estimate, offload not measured | CPU offload (slow) |
| 120B MoE (5.1B active) | est. 14 tok/srough estimate, offload not measured | CPU offload (slow) |
Decode speed, generic Q4_K_M model of each size at 4K context, 732 GB/s, typical error 5-15%
ModelFit estimates, not measured benchmarks: decode speed of a generic Q4_K_M model of each size at 4K context on the Tesla P100's 732 GB/s bandwidth. MoE rows scale by active parameters (decode reads only the active experts), so a 35B MoE runs far faster than a dense 32B. "CPU offload" sizes exceed the 16GB VRAM; dense models slow to a crawl there, MoE models degrade less because hot experts stay GPU-resident. Offload rows are rough estimates: no measured run covers offload, so they carry no typical error. What typical error means.
Mellum2.1 12B-A2.5B Thinking loads ~8.1 GB of weights; at 16k context the KV cache adds ~0.2 GB (still fits the ~14 GB usable VRAM), and at 64k it adds ~0.9 GB (still fits).
The model leaves about 6.1 GB of the usable VRAM budget free after weights and 16k context.
KV-cache figures assume an fp16 cache, the llama.cpp/Ollama default. Standard GQA models use a size-class estimate (8 KV heads x 128 head dim class); hybrid linear-attention models (Qwen3.5/3.6, Qwen3-Next) use the exact per-token cost from their published config, since only their sparse full-attention layers cache KV. A q8_0 KV cache roughly halves either figure. Estimates, not measurements.
One email when a new open-weight model is a better fit for 16 GB of VRAM, plus the Thursday weekly.
You'll also get the weekly ModelFit email. By signing up you agree to our Privacy Policy. Unsubscribe from either email anytime, in one click.
ModelFit may earn a commission on purchases through these links, at no extra cost to you.
ModelFit budgets about 14.4GB of the 16GB for weights and context, so 14B models at Q4 (~11GB) load with room for context. Its 732 GB/s is more raw bandwidth than the RTX 5060 Ti (448 GB/s), but Pascal and HBM cards decode below their raw bandwidth in the measured runs the speed model is fitted on, so the estimate is est. 42 tok/s, typical error 5% against est. 68 tok/s on 8B. Prompt reading is the weak side: its FP16 rate is 18.7 TFLOPS with no tensor cores, a fraction of an RTX card, which the first-token times below reflect.
| Hardware | Memory | Speed | Bandwidth |
|---|---|---|---|
| RTX 3060 | 12 GB | est. 50 tok/stypical error 5% | 360 GB/s |
| Tesla P100 | 16 GB | est. 42 tok/stypical error 5% | 732 GB/s |
| RTX 4060 Ti | 16 GB | est. 43 tok/stypical error 5% | 288 GB/s |
| RTX 5060 Ti | 16 GB | est. 68 tok/stypical error 5% | 448 GB/s |
| RTX 3090 | 24 GB | est. 117 tok/stypical error 5% | 936 GB/s |
Best for: Agentic coding, Small machines·Pop: 76/100
Perf: est. 84 tok/s, typical error 15% · first token ~0.5s
Fits in 16 GB VRAM with room to spare. Best for agentic coding, small machines on Tesla P100.
Best for: Chat, Coding, Multimodal·Pop: 80/100
Perf: est. 30 tok/s, typical error 15% · first token ~2.4s
Fits in 16 GB VRAM with room to spare. Best for chat, coding, multimodal on Tesla P100.
Best for: Voice assistants, Tool calling, Chat·Pop: 75/100
Perf: est. 57 tok/s, typical error 5% · first token ~1.5s
Fits in 16 GB VRAM with room to spare. Best for voice assistants, tool calling, chat on Tesla P100.
Best for: Quality, Coding, Reasoning·Pop: 86/100
Perf: est. 28 tok/s, typical error 5% · first token ~10.5s
Fits in 16 GB VRAM with room to spare. Best for quality, coding, reasoning on Tesla P100.
Best for: On-device, Mobile, Chat·Pop: 82/100
Perf: est. 45 tok/s, typical error 5% · first token ~9.3s
Fits in 16 GB VRAM with room to spare. Best for on-device, mobile, chat on Tesla P100.
Best for: Chat, Coding, Multimodal·Pop: 80/100
Perf: est. 18 tok/s, typical error 15% · first token ~15.0s
Fits in 16 GB VRAM with room to spare. Best for chat, coding, multimodal on Tesla P100.
Best for: Local AI agents, privacy-first tool calling, MCP workflows·Pop: 80/100
Perf: est. 93 tok/s, typical error 15% · first token ~0.5s
Fits in 16 GB VRAM with room to spare. Best for local ai agents, privacy-first tool calling, mcp workflows on Tesla P100.
Best for: Quality, Coding, Reasoning·Pop: 86/100
Perf: est. 45 tok/s, typical error 5% · first token ~1.7s
Fits in 16 GB VRAM with room to spare. Best for quality, coding, reasoning on Tesla P100.
Best for: Agentic coding on small machines·Pop: 76/100
Perf: est. 45 tok/s, typical error 5% · first token ~1.7s
Fits in 16 GB VRAM with room to spare. Best for agentic coding on small machines on Tesla P100.
Best for: On-device agents, tool calling, multilingual chat·Pop: 72/100
Perf: est. 119 tok/s, typical error 15% · first token ~0.4s
Fits in 16 GB VRAM with room to spare. Best for on-device agents, tool calling, multilingual chat on Tesla P100.
Runs · Stated: the linked source says a GPU with that much VRAM runs the model. Runs · Published: a dated run on the named card with that much VRAM. unknown: no source yet. Fits with offload: the main model weights fit in VRAM with a 15% margin; the text encoder and VAE run from system RAM, which is slower. Exceeds VRAM: even the main weights are too large for 16 GB. Every VRAM tier: video · image. Ranked with reasons: video models for 16 GB of VRAM · image models for 16 GB of VRAM.
| Model | Fits | Quality | App |
|---|---|---|---|
| HunyuanVideo-1.5Tencent Hunyuan Community License | Runs · Stated14 GB VRAM · official runner with model offloading · HunyuanVideo-1.5 model card | #38 of 48 in text to video4,292 votessnapshot 2026-09-22 | ComfyUI |
| LTX-2.3LTX-2 Community License Agreement | Runs · Stated16 GB VRAM · distilled with FP8 cast, previews only · LTX blog: hardware for AI video models | No independent score yet | Lightricks/LTX-2 ltx-pipelines, ComfyUI |
| LTX-2.5LTX-2.x Community License Agreement | Runs · Stated12 GB VRAM · LTX Desktop on Windows, weight streaming, 40 GB RAM, 540p · LTX Desktop compatibility matrix (Lightricks) | No independent score yet | Lightricks/LTX-2 ltx-pipelines, ComfyUI |
| MiniMax H3MiniMax H3 Community License Agreement | Runs · PublishedRTX 5060 · 16 GB VRAM · ComfyUI int8 convrot with int4 text encoder, 1024x768, 10 s in 24 min · ComfyUI issue #15274 | #8 of 48 in text to video11,417 votessnapshot 2026-09-22 | stable-diffusion.cpp, ComfyUI |
| Wan2.1Apache License 2.0 | Runs · Stated13 GB VRAM · T2V 14B in Diffusers, bf16 with group offloading to system RAM · Diffusers docs: Wan pipeline | No independent score yet | ComfyUI |
| Wan2.2 A14BApache License 2.0 | Runs · PublishedRTX 5060 Ti · 16 GB VRAM · ComfyUI fp16 I2V with 4-step LoRAs, 64 GB system RAM · ComfyUI issue #10662 | #42 of 48 in text to video10,458 votessnapshot 2026-09-22 | ComfyUI |
| Wan2.2 TI2V 5BApache License 2.0 | Runs · Stated8 GB VRAM · ComfyUI native offloading · ComfyUI docs: Wan2.2Complete: Comfy-Org repackaged FP16 with FP8 scaled UMT518.1 GB download | No independent score yet | ComfyUI |
| Mochi 1 previewApache License 2.0 | unknownComplete: calcuis GGUF Q3_K_M with Q4_0 T5 and FP8 VAE7.7 GB download | #48 of 48 in text to video5,893 votessnapshot 2026-09-22 | Diffusers, ComfyUI |
Quality: blind human votes from Arena (arena.ai), CC BY 4.0, snapshot 2026-09-22. Scores are for the Arena checkpoint, not your local quantized build.
| Model | Fits | Quality | App |
|---|---|---|---|
| FLUX 3 Image KleinFLUX Kommunity License v1.0 | Runs · Stated8 GB VRAM · official ComfyUI int8 files · FLUX 3 Image Klein README (Black Forest Labs) | No independent score yet | Diffusers |
| FLUX.1 [dev]FLUX.1 [dev] Non-Commercial License v1.1.1 | Runs · Stated6 GB VRAM · stable-diffusion.cpp with GGUF weights · stable-diffusion.cpp FLUX docs | #78 of 82 in text to image49,221 votessnapshot 2026-10-07 | Diffusers, ComfyUI |
| FLUX.2 [klein]Apache 2.0 (4B), FLUX Non-Commercial License (9B) | Runs · Stated13 GB VRAM · Klein 4B · FLUX.2 [klein] 4B model cardComplete: Klein 4B BF1616.0 GB download | No independent score yet | Diffusers, stable-diffusion.cpp |
| Krea 2 TurboKrea 2 Community License Agreement v.1 | Runs · PublishedRTX 4070 · 12 GB VRAM · ComfyUI fp8, dynamic VRAM paging, 8 steps · ComfyUI issue #15365 | No independent score yet | Diffusers, stable-diffusion.cpp |
| Qwen-Image-2.1Qwen Research License Agreement | Runs · PublishedRTX 5080 · 16 GB VRAM · stable-diffusion.cpp Q8_0, 1024x1024 · stable-diffusion.cpp issue #2041 | #19 of 82 in text to image11,441 votessnapshot 2026-10-07 | Diffusers, stable-diffusion.cpp |
| SDXL Base 1.0CreativeML Open RAIL++-M License | Runs · Stated8 GB VRAM · SDXL 1.0, no app named · SDXL 1.0 announcement (Stability AI) | No independent score yet | ComfyUI, stable-diffusion.cpp |
| Z-Image-TurboApache 2.0 | Runs · Stated16 GB VRAM · stated for consumer GPUs · Z-Image-Turbo model card | #59 of 82 in text to image24,808 votessnapshot 2026-10-07 | Diffusers, stable-diffusion.cpp |
| MOSS-ImageApache 2.0 | unknownComplete: UD-Q4_K_XL GGUF13.2 GB download | No independent score yet | stable-diffusion.cpp |
| Stable Diffusion 3.5 LargeStability AI Community License | Fits with offloadComplete: Q4_0 GGUF16.4 GB download | #81 of 82 in text to image23,396 votessnapshot 2026-10-07 | ComfyUI |
| FLUX.2 [dev]FLUX Non-Commercial License | Exceeds VRAMComplete: Q4_K_M GGUF34.8 GB download | #38 of 82 in text to image77,988 votessnapshot 2026-10-07 | Diffusers, ComfyUI, stable-diffusion.cpp |
Quality: blind human votes from Arena (arena.ai), CC BY 4.0, snapshot 2026-10-07. Scores are for the Arena checkpoint, not your local quantized build.
Speeds are estimates for Qwen3 8B Q4_K_M, typical error 5%.
The Tesla P100 PCIe has 16GB of HBM2 at 732 GB/s (NVIDIA datasheet). ModelFit budgets about 14.4GB of it for weights and context, enough for dense models up to 14B at Q4 quantization.
Yes. Ollama lists the Tesla P100 under compute capability 6.0 in its NVIDIA support table. Cards of compute capability 5.0 to 6.2 need NVIDIA driver version 570 or newer.
Dense models up to 14B at Q4, such as Qwen3 14B (~11GB) at est. 25 tok/s, typical error 15%. 8B models run at est. 42 tok/s, typical error 5% with room for a long context. 27B and 32B dense models spill into system RAM.
Both hold 16GB. The 5060 Ti is estimated at est. 68 tok/s, typical error 5% on 8B against est. 42 tok/s for the P100, and its tensor cores read prompts much faster. The P100 is a passively cooled 250 W server card (NVIDIA datasheet) that needs case airflow built for it; the 5060 Ti is a desktop card with its own fans.
The Tesla P100's 16GB of VRAM cannot fit a 32B model comfortably. The largest size class it fits is 20B MoE (3.6B active), at est. 57 tok/s, typical error 15%. The largest individual model in the catalog that fits is LFM2 24B-A2B Instruct (24B).
Time a model that fits your machine with Ollama, using the same prompts as every other run. The CLI shows your result and asks before it sends anything; your machine then joins the speed test leaderboard. It does not change the quality benchmark.
Use our interactive wizard to compare models across Apple Silicon and NVIDIA GPUs.