DeepSeek-R1 Distill Qwen 7B quants compared
24 GGUF builds by real file size, probed from bartowski/DeepSeek-R1-Distill-Qwen-7B-GGUF on Hugging Face (2026-09-02). 7B params.
Download DeepSeek-R1 Distill Qwen 7B Q4_K_M (4.36 GB) — it fits 8 GB of memory with 16k context. With Ollama: ollama run deepseek-r1:7b
Every DeepSeek-R1 Distill Qwen 7B quant by real file size
| Quant | Weights | + KV (16k) | Total | Fits comfortably in | Quality |
|---|---|---|---|---|---|
| F32 | 28.38 GB | 2.0 GB | 30.4 GB | 48 GB | Full precision (lossless) |
| F16 | 14.19 GB | 2.0 GB | 16.2 GB | 24 GB | Full precision (lossless) |
| Q8_0 | 7.54 GB | 2.0 GB | 9.5 GB | 12 GB | Near-lossless |
| Q6_K_L | 6.07 GB | 2.0 GB | 8.1 GB | 12 GB | Excellent |
| Q6_K | 5.82 GB | 2.0 GB | 7.8 GB | 12 GB | Excellent |
| Q5_K_L | 5.38 GB | 2.0 GB | 7.4 GB | 12 GB | Very high |
| Q5_K_M | 5.07 GB | 2.0 GB | 7.1 GB | 8 GB | Very high |
| Q5_K_S | 4.95 GB | 2.0 GB | 7.0 GB | 8 GB | Very high |
| Q4_K_M * | 4.36 GB | 2.0 GB | 6.4 GB | 8 GB | High — the default pick |
| Q4_K_L | 4.74 GB | 2.0 GB | 6.7 GB | 8 GB | High |
| Q4_1 | 4.54 GB | 2.0 GB | 6.5 GB | 8 GB | High |
| Q4_K_S | 4.15 GB | 2.0 GB | 6.2 GB | 8 GB | High |
| Q4_0 | 4.14 GB | 2.0 GB | 6.1 GB | 8 GB | High |
| IQ4_NL | 4.13 GB | 2.0 GB | 6.1 GB | 8 GB | High |
| IQ4_XS | 3.93 GB | 2.0 GB | 5.9 GB | 8 GB | High |
| Q3_K_XL | 4.25 GB | 2.0 GB | 6.3 GB | 8 GB | Acceptable — visible loss |
| Q3_K_L | 3.81 GB | 2.0 GB | 5.8 GB | 8 GB | Acceptable — visible loss |
| Q3_K_M | 3.55 GB | 2.0 GB | 5.5 GB | 8 GB | Acceptable — visible loss |
| IQ3_M | 3.33 GB | 2.0 GB | 5.3 GB | 8 GB | Acceptable — visible loss |
| Q3_K_S | 3.25 GB | 2.0 GB | 5.3 GB | 8 GB | Acceptable — visible loss |
| IQ3_XS | 3.12 GB | 2.0 GB | 5.1 GB | 8 GB | Acceptable — visible loss |
| Q2_K_L | 3.3 GB | 2.0 GB | 5.3 GB | 8 GB | Experimental — not ranked — never recommended |
| Q2_K | 2.81 GB | 2.0 GB | 4.8 GB | 8 GB | Experimental — not ranked — never recommended |
| IQ2_M | 2.59 GB | 2.0 GB | 4.6 GB | 8 GB | Experimental — not ranked — never recommended |
* default pick. Weights = real GGUF file sizes from bartowski/DeepSeek-R1-Distill-Qwen-7B-GGUF (probed 2026-09-02). KV = fp16 estimate; a q8_0 cache roughly halves it. "Comfortable" = weights + KV within 90% of memory.
Best DeepSeek-R1 Distill Qwen 7B quant by memory
| Memory | Recommended quant | Total (16k ctx) |
|---|---|---|
| 8 GB | Q5_K_M | 7.1 GB |
| 12 GB | Q8_0 | 9.5 GB |
| 24 GB | F16 | 16.2 GB |
Why we don't rank DeepSeek-R1 Distill Qwen 7B's 2-bit quants
Quants at 2 bits per weight or below (Q2_K, IQ2, IQ1, TQ1) cut file size by roughly half versus Q4, but the quality collapse is steep and non-linear: perplexity spikes, instruction-following degrades, and hallucinations rise. A model that answers faster but wrong is not a smaller model — it is a worse one. ModelFit lists these builds for completeness but never ranks or recommends them.
Frequently asked questions
What is the best quantization of DeepSeek-R1 Distill Qwen 7B?
Q4_K_M is the default pick: 4.36 GB of weights, high — the default pick quality, fitting comfortably in 8 GB of memory (weights + 16k context KV-cache). Go Q6_K or Q8_0 if you have headroom.
How much memory does DeepSeek-R1 Distill Qwen 7B need?
At Q4_K_M, DeepSeek-R1 Distill Qwen 7B needs 4.36 GB for the weights plus ~2.0 GB of KV-cache at 16k context — about 6.4 GB total, so a 8 GB card or Mac (90% usable budget) runs it comfortably.
Should I use a Q2_K or IQ2 quant of DeepSeek-R1 Distill Qwen 7B?
No. DeepSeek-R1 Distill Qwen 7B at 2 bits per weight is a visibly worse model — quality collapse at that bitrate is steep, not gradual. If only a 2-bit build fits your memory, run a smaller model at Q4_K_M instead. ModelFit lists these builds but never recommends them.
Cite this page
ModelFit: DeepSeek-R1 Distill Qwen 7B quantization comparison (real GGUF file sizes). https://modelfit.io/quant-compare/deepseek-r1-distill-qwen-7b/ (data probed 2026-09-02, CC BY 4.0).