VRAM Calculator for Local LLMs

Budget about 0.6 GB of VRAM per billion parameters at Q4 quantization for the weights, and treat roughly 90% of the card as usable so the display and the context cache still have room. In practice an 8GB card runs up to an 8B model, 12GB reaches 9B, 16GB unlocks 14B-class models, 24GB runs 32B-class models, and 32GB reaches 70B-class mixture-of-experts builds that keep only a few billion parameters active per token.

The card also has to hold the CUDA or Metal context, the KV cache for your conversation, and whatever your desktop is displaying. ModelFit budgets 90% of VRAM for model weights so the fit test is honest on a real machine rather than matching a spec sheet. On Apple Silicon the equivalent budget is 70% of unified memory up to 32GB, rising to about 85% at 128GB and above, because the GPU shares that memory with macOS.

Yes, and it is the most common surprise. The KV cache grows linearly with context length, from about 112 KB per 1,000 tokens on small models to over 320 KB per 1,000 tokens on a 70B model with an fp16 cache. A 70B model at 64k tokens of context therefore needs roughly another 20GB on top of its 42GB of weights. Quantizing the KV cache to q8_0 roughly halves that, and hybrid linear-attention models such as Qwen3.5 cache only a fraction of what a classic attention layout needs.

Not at full speed, and not as a dense model. A dense 70B at Q4 needs about 42GB of weights, which exceeds the 22GB usable budget of a 24GB card. The workable paths are a mixture-of-experts model of that total size with a few billion active parameters, a smaller quantization with some layers offloaded to system RAM, or pooling two cards. Pooled fit is what the multi-GPU rig mode of the calculator computes; expect throughput below a single card with the same total VRAM because layer transfers add overhead.