LFM2 24B-A2B Instruct quants compared
27 GGUF builds by real file size, probed from bartowski/LiquidAI_LFM2-24B-A2B-GGUF on Hugging Face (2026-09-02). 24B params, 2B active.
Download LFM2 24B-A2B Instruct Q4_K_M (13.44 GB) — it fits 24 GB of memory with 16k context. With Ollama: ollama run lfm2:24b-a2b
Every LFM2 24B-A2B Instruct quant by real file size
| Quant | Weights | + KV (16k) | Total | Fits comfortably in | Quality |
|---|---|---|---|---|---|
| BF16 | 44.42 GB | 4.0 GB | 48.4 GB | 64 GB | Full precision (lossless) |
| Q8_0 | 23.61 GB | 4.0 GB | 27.6 GB | 32 GB | Near-lossless |
| Q6_K_L | 18.27 GB | 4.0 GB | 22.3 GB | 32 GB | Excellent |
| Q6_K | 18.24 GB | 4.0 GB | 22.2 GB | 32 GB | Excellent |
| Q5_K_L | 15.8 GB | 4.0 GB | 19.8 GB | 24 GB | Very high |
| Q5_K_M | 15.77 GB | 4.0 GB | 19.8 GB | 24 GB | Very high |
| Q5_K_S | 15.31 GB | 4.0 GB | 19.3 GB | 24 GB | Very high |
| Q4_K_M * | 13.44 GB | 4.0 GB | 17.4 GB | 24 GB | High — the default pick |
| Q4_1 | 13.93 GB | 4.0 GB | 17.9 GB | 24 GB | High |
| Q4_K_L | 13.47 GB | 4.0 GB | 17.5 GB | 24 GB | High |
| Q4_K_S | 12.99 GB | 4.0 GB | 17.0 GB | 24 GB | High |
| Q4_0 | 12.77 GB | 4.0 GB | 16.8 GB | 24 GB | High |
| IQ4_NL | 12.56 GB | 4.0 GB | 16.6 GB | 24 GB | High |
| IQ4_XS | 11.87 GB | 4.0 GB | 15.9 GB | 24 GB | High |
| Q3_K_XL | 10.53 GB | 4.0 GB | 14.5 GB | 24 GB | Acceptable — visible loss |
| Q3_K_L | 10.5 GB | 4.0 GB | 14.5 GB | 24 GB | Acceptable — visible loss |
| Q3_K_M | 10.1 GB | 4.0 GB | 14.1 GB | 16 GB | Acceptable — visible loss |
| IQ3_M | 10.09 GB | 4.0 GB | 14.1 GB | 16 GB | Acceptable — visible loss |
| Q3_K_S | 9.64 GB | 4.0 GB | 13.6 GB | 16 GB | Acceptable — visible loss |
| IQ3_XS | 9.11 GB | 4.0 GB | 13.1 GB | 16 GB | Acceptable — visible loss |
| IQ3_XXS | 8.75 GB | 4.0 GB | 12.8 GB | 16 GB | Acceptable — visible loss |
| Q2_K_L | 7.78 GB | 4.0 GB | 11.8 GB | 16 GB | Experimental — not ranked — never recommended |
| Q2_K | 7.75 GB | 4.0 GB | 11.8 GB | 16 GB | Experimental — not ranked — never recommended |
| IQ2_M | 7.21 GB | 4.0 GB | 11.2 GB | 16 GB | Experimental — not ranked — never recommended |
| IQ2_S | 6.45 GB | 4.0 GB | 10.4 GB | 12 GB | Experimental — not ranked — never recommended |
| IQ2_XS | 6.17 GB | 4.0 GB | 10.2 GB | 12 GB | Experimental — not ranked — never recommended |
| IQ2_XXS | 5.35 GB | 4.0 GB | 9.3 GB | 12 GB | Experimental — not ranked — never recommended |
* default pick. Weights = real GGUF file sizes from bartowski/LiquidAI_LFM2-24B-A2B-GGUF (probed 2026-09-02). KV = fp16 estimate; a q8_0 cache roughly halves it. "Comfortable" = weights + KV within 90% of memory.
Best LFM2 24B-A2B Instruct quant by memory
| Memory | Recommended quant | Total (16k ctx) |
|---|---|---|
| 16 GB | Q3_K_M | 14.1 GB |
| 24 GB | Q5_K_M | 19.8 GB |
| 32 GB | Q8_0 | 27.6 GB |
| 64 GB | BF16 | 48.4 GB |
Why we don't rank LFM2 24B-A2B Instruct's 2-bit quants
Quants at 2 bits per weight or below (Q2_K, IQ2, IQ1, TQ1) cut file size by roughly half versus Q4, but the quality collapse is steep and non-linear: perplexity spikes, instruction-following degrades, and hallucinations rise. A model that answers faster but wrong is not a smaller model — it is a worse one. ModelFit lists these builds for completeness but never ranks or recommends them.
Frequently asked questions
What is the best quantization of LFM2 24B-A2B Instruct?
Q4_K_M is the default pick: 13.44 GB of weights, high — the default pick quality, fitting comfortably in 24 GB of memory (weights + 16k context KV-cache). Go Q6_K or Q8_0 if you have headroom.
How much memory does LFM2 24B-A2B Instruct need?
At Q4_K_M, LFM2 24B-A2B Instruct needs 13.44 GB for the weights plus ~4.0 GB of KV-cache at 16k context — about 17.4 GB total, so a 24 GB card or Mac (90% usable budget) runs it comfortably.
Should I use a Q2_K or IQ2 quant of LFM2 24B-A2B Instruct?
No. LFM2 24B-A2B Instruct at 2 bits per weight is a visibly worse model — quality collapse at that bitrate is steep, not gradual. If only a 2-bit build fits your memory, run a smaller model at Q4_K_M instead. ModelFit lists these builds but never recommends them.
Cite this page
ModelFit: LFM2 24B-A2B Instruct quantization comparison (real GGUF file sizes). https://modelfit.io/quant-compare/lfm2-24b-a2b/ (data probed 2026-09-02, CC BY 4.0).