GPU VRAM Calculator

Eight gigabytes still covers the small-model tier. The 90% usable budget leaves about 7 GB for weights, which holds an 8B model at Q4 (5 GB) with modest context and powers lightweight chat and coding assistants; the RTX 4060 row of the table above shows exactly what the engine pairs with it. A 14B model needs about 8 GB of weights, so an 8GB card only reaches it with heavier quantization or CPU offload, both of which cost quality and speed. If 14B-class is your floor, a 12GB or 16GB card is the more comfortable starting point.

They answer different questions. VRAM decides what fits: nothing runs if the weights do not fit the budget. Bandwidth decides how fast it runs once it does, because decode speed is roughly proportional to it for a given model size, which is why every tok/s figure on this page starts from it (bytes read per token over bandwidth, with efficiencies fitted on measured runs). The practical consequence is that a faster older card with less VRAM can beat a roomier one on small models, so the best-pick column weighs fit first and speed second. Mixture-of-experts models bend the rule in your favor: only their active experts are read per token, so decode speed tracks active parameters, not the total.

A dense 32B at Q4 needs about 19 GB of weights, and against the 90% budget that implies a card with at least 22 GB of VRAM. In practice that means a 24GB card, such as the RTX 3090, RTX 4090 or RX 7900 XTX, which all hold it comfortably per the table above. A 16GB card lands at about 14 GB usable, roughly 5 GB short of a dense 32B, so it falls back to lower quantizations or CPU offload.

Yes, and that is exactly what the multi-GPU rig mode of the calculator below computes. Two 16GB cards pool about 29 GB of usable weight budget, which clears the 19 GB a dense 32B needs. The honest caveat is that pooled throughput is lower than a single card with the same total VRAM, because layers are transferred between cards, and mixture-of-experts models tolerate pooling better since only their active experts cross the link. Pooling is the practical path to 70B-class models on consumer cards, which is why the rig mode exists.