Gemma 4 is a family, not a model, and its VRAM requirements span a factor of ten. The E2B edge variant loads in 2.3 GB while the 31B dense flagship wants 20 GB. Picking the wrong variant for your hardware is the most common Gemma 4 mistake. This guide maps all five variants to the GPUs and Macs that actually run them, with engine-estimated speeds.
TL;DR: E2B (2.3 GB) runs on anything including phones. E4B (4 GB) is the 8GB-GPU pick. 12B (8 GB) is the sweet spot for 12-16GB GPUs and 16GB Macs. 26B-A4B (16 GB, MoE) wants 24GB. 31B (20 GB, dense) is the quality flagship for 24-32GB. Every variant except E2B is a strong local model in 2026.
VRAM Requirements by Variant
All sizes are Q4_K_M as shipped via Ollama. These are the builds ModelFit tracks.
| Variant | Type | Loaded size | Min practical VRAM/RAM | Best for |
|---|---|---|---|---|
| Gemma 4 E2B | Dense 2.3B | ~2.3 GB | 4 GB | Edge, mobile, IoT |
| Gemma 4 E4B | Dense 4.5B | ~4 GB | 6-8 GB | On-device, 8GB GPUs |
| Gemma 4 12B | Dense 12B | ~8 GB | 12 GB | The all-rounder |
| Gemma 4 26B-A4B | MoE 26B (4B active) | ~16 GB | 24 GB | Multimodal, fast MoE |
| Gemma 4 31B | Dense 31B | ~20 GB | 32 GB | Maximum quality |
Context adds on top. The 12B and 31B are dense GQA models with standard KV cache costs, roughly 128-192 KB per token at fp16. The 26B-A4B is multimodal and its mixture-of-experts design decodes at the speed of a 4B model while occupying 16 GB.
GPU Fit by Variant
Speeds are ModelFit engine estimates at Q4_K_M, labeled est.
Gemma 4 12B (the most searched)
| GPU | VRAM | Est. tok/s | Verdict |
|---|---|---|---|
| RTX 5090 32GB | 32 GB | ~103 est. | Overkill |
| RTX 4090 24GB | 24 GB | ~74 est. | Excellent |
| RTX 5080 16GB | 16 GB | ~67 est. | Excellent |
| RTX 5070 12GB | 12 GB | ~42 est. | Comfortable |
| RTX 3060 12GB | 12 GB | ~30 est. | Comfortable |
| RTX 4060 8GB | 8 GB | ~4 est. | Too tight |
On Macs, the 12B is the flagship pick for 16GB machines: ~19 tok/s est. on a Mac Mini M6 16GB, ~17 tok/s est. on a fanless MacBook Air M5 16GB.
Gemma 4 26B-A4B and 31B
| GPU | VRAM | 26B-A4B est. | 31B est. |
|---|---|---|---|
| RTX 5090 32GB | 32 GB | ~118 | ~46 |
| RTX 4090 24GB | 24 GB | ~85 | ~33 |
| RTX 3090 24GB | 24 GB | ~71 | ~28 |
| RX 7900 XTX 24GB | 24 GB | ~72 | ~28 |
| RTX 5080 16GB | 16 GB | ~57 (tight) | Offload, avoid |
The 26B-A4B is the faster of the two on the same card because its 4B active path reads far fewer bytes per token. On 24GB it runs about twice as fast as the dense 31B while scoring within a point of it on quality benchmarks. For most buyers, the MoE is the right 24GB pick.
Which Variant Should You Pick?
- 8GB GPU (RTX 4060, 3070): E4B at ~49 tok/s est., or Qwen 3.5 4B if you want a sharper small model. The 12B does not fit.
- 12GB GPU (RTX 3060, 5070): 12B is the headline. ~30-42 tok/s est.
- 16GB GPU (RTX 5060 Ti, 5080): 12B with room for context, or 26B-A4B tight. Prefer 24GB for the MoE.
- 24GB GPU (RTX 3090, 4090): 26B-A4B for speed or 31B for maximum dense quality.
- 16GB Mac: 12B at ~17-19 tok/s est. is the best current-gen fit.
- 24GB Mac: 26B-A4B at ~21 tok/s est. on an M6 Mac Mini.
- 48GB+ Mac: 31B at ~13 tok/s est. on an M5 Pro Mac Mini 48GB, or step up to Qwen 3.6 35B-A3B.
FAQ
Is Gemma 4 12B enough for coding?
Yes. It scores well on LiveCodeBench and is our top pick for a 16GB Mac. For dedicated coding agents, Ornith 1.0 9B is faster at the same memory footprint, but Gemma 4 12B is the better general model.
What is the difference between 26B-A4B and 31B?
Architecture. The 31B is a traditional dense model. The 26B-A4B is a mixture of experts with 4B active per token, so it runs roughly twice as fast on the same hardware with nearly the same quality. Unless you specifically want dense weights, the MoE is the better 24GB choice.
Does Gemma 4 E2B run on a phone?
Yes. At 2.3 GB it is designed for on-device use and runs on recent iPhones and Android flagships. It is the weakest Gemma 4 by a wide margin, so treat it as an edge model, not a desktop assistant.
Why does the 12B not fit on an 8GB GPU?
The weights are 8 GB before the OS, display server, and context cache. On an 8GB card the model spills into shared system memory and collapses to single-digit tokens per second. On Windows the problem is worse because the OS reserves more VRAM. Use E4B on 8GB.
Match this model to a machine that can run it: by RAM tier for Apple Silicon, or by VRAM for an NVIDIA GPU.
The weekly local-AI refresh
New open-weight models, real Apple Silicon benchmarks, and the one model worth running on your Mac this week. Free, one email a week, unsubscribe anytime.
By subscribing you agree to our Privacy Policy and to receive the weekly email. Unsubscribe anytime.
Have questions? Reach out on X/Twitter