VRAM for a 70B Model

Dense 70B-class models in the ModelFit catalog, one row per quantized build. Totals add the KV cache at 8k and 32k of context to the weights.

KV cache: 2.5 GB at 8k and 10 GB at 32k per model, a size-class estimate (no exact per-token figure in the catalog). Speeds are ModelFit estimates, not measured runs. Updated 2026-10-31.

The engine's verdict for the 42 GB build on each memory size, smallest first, with the longest context (8k to 128k) that still fits next to the weights.

Verdicts and speeds come from the same engine as each GPU page and the calculator. A pooled rig is a fit check only (the rig calculator rule: 90% of the combined VRAM); split speed depends on the cards and the link between them, so none is estimated. A Mac row uses the engine's machine for that memory size.

How these numbers are made

• Sizes: the loaded size and minimum memory of each build from the open dataset (/api/dataset/, CC BY 4.0). Parameter count from the publisher's model card (Llama 3.3 70B Instruct).
• Context: KV-cache figures assume an fp16 cache, the llama.cpp/Ollama default. Standard GQA models use a size-class estimate (8 KV heads x 128 head dim class); hybrid linear-attention models (Qwen3.5/3.6, Qwen3-Next) use the exact per-token cost from their published config, since only their sparse full-attention layers cache KV. A q8_0 KV cache roughly halves either figure. Estimates, not measurements.
• Fit: 90% of a card's VRAM, 70% to 85% of a Mac's unified memory, then the engine's verdict, as in the calculator.
• Speed: decode tokens per second at 4k of context from the fitted bandwidth model, each with its typical error (the median gap to measured runs on hardware the model was not fitted on). A row with CPU offload has no measured band. How the estimates work.