Local LLM Compatibility Dataset

172 models by params, quantization, and memory load. Which run locally on Apple Silicon and NVIDIA GPUs. Free under CC BY 4.0. Updated 2026-10-26.

JSON exportGitHubHugging Face
Models tracked
172
Run locally
129
License
CC BY 4.0
Updated
2026-10-26

This is ModelFit's open compatibility dataset: every model the site tracks, with the parameter size, quantization, minimum RAM, and estimated memory load used to decide what runs locally. Memory figures are system/unified RAM; the same budget math applies to GPU VRAM (per-card breakdowns live in the GPU guides). Reuse it freely with attribution (CC BY 4.0). Credit ModelFit (modelfit.io). Machine-readable version: /api/dataset/, or get the CSV + JSON on GitHub and Hugging Face (load_dataset ready).

Prefer the terminal? The same dataset and engine power the open-source CLI: npx @modelfit/cli@latest names the best local model for the machine it runs on. Get it on npm or GitHub.

Table 1Compatibility matrix
172 of 172 models
FamilyQuantLocalollama
AREX-2 27BAREX27.4BQ4_K_M32 GB~17.2 GBYesn/a
Beam 1.0Beam501BAPI0 GBn/aOpen, no fitn/a
Claude 3.5 SonnetClaudeUndisclosedAPI0 GBn/aCloudn/a
Claude 3.7 SonnetClaudeUndisclosedAPI0 GBn/aCloudn/a
Claude 3 OpusClaudeUndisclosedAPI0 GBn/aCloudn/a
Claude 4 OpusClaudeUndisclosedAPI0 GBn/aCloudn/a
Claude 4 SonnetClaudeUndisclosedAPI0 GBn/aCloudn/a
Claude Opus 4.7ClaudeUndisclosedAPI0 GBn/aCloudn/a
Claude Opus 4.8ClaudeUndisclosedAPI0 GBn/aCloudn/a
Claude Fable 5ClaudeUndisclosedAPI0 GBn/aCloudn/a
DeepSeek-R1 Distill Qwen 7BDeepSeek7BQ4_K_M8 GB~5.5 GBYesdeepseek-r1:7b
DeepSeek-R1 Distill Qwen 7B (Q8)DeepSeek7BQ8_016 GB~7.5 GBYesdeepseek-r1:7b-qwen-distill-q8_0
DeepSeek-R1 Distill Qwen 14BDeepSeek14BQ4_K_M16 GB~11 GBYesdeepseek-r1:14b
DeepSeek-R1 Distill Qwen 14B (Q8)DeepSeek14BQ8_024 GB~14.6 GBYesdeepseek-r1:14b-qwen-distill-q8_0
DeepSeek-R1 Distill Llama 70BDeepSeek70BQ4_K_M64 GB~42 GBYesdeepseek-r1:70b
DeepSeek V4 Flash 0731DeepSeek284BAPI0 GBn/aOpen (llama.cpp)n/a
DeepSeek V4.1 FlashDeepSeek552BAPI0 GBn/aOpen, no fitn/a
DeepSeek-R1 671BDeepSeek671BQ4_K_M512 GB~380 GBYesdeepseek-r1:671b-q4_K_M
DeepSeek-V3DeepSeek671BAPI0 GBn/aOpen, no fitn/a
DeepSeek-V3-0324DeepSeek671BAPI0 GBn/aOpen, no fitn/a
DeepSeek-R1DeepSeek671BAPI0 GBn/aOpen, no fitn/a
DeepSeek V4 ProDeepSeek1600BAPI0 GBn/aOpen, no fitn/a
Flint 40B-A2.5BFlint39.8BNVFP436 GB~22.9 GBYesflint:40b
Gemma 3 1B InstructGemma1BQ4_K_M2 GB~1 GBYesgemma3:1b
Gemma 3 1B Instruct (Q8)Gemma1BQ8_04 GB~1 GBYesgemma3:1b-it-q8_0
Gemma 2 2B InstructGemma2BQ4_K_M4 GB~1.8 GBYesgemma2:2b-instruct-q4_K_M
Gemma 4 E2BGemma2.3BQ4_K_M4 GB~2.3 GBYesgemma4:e2b
Gemma 4 E2B (Q8)Gemma2.3BQ8_08 GB~4.6 GBYesgemma4:e2b-it-q8_0
Gemma 3 4B InstructGemma4BQ4_K_M6 GB~3.5 GBYesgemma3:4b
Gemma 3 4B Instruct (Q8)Gemma4BQ8_08 GB~3.9 GBYesgemma3:4b-it-q8_0
Gemma 4 E4BGemma4.5BQ4_K_M6 GB~4 GBYesgemma4:e4b
Gemma 4 E4B (Q8)Gemma4.5BQ8_016 GB~7.5 GBYesgemma4:e4b-it-q8_0
Gemma 2 9B InstructGemma9BQ4_K_M12 GB~7 GBYesgemma2:9b-instruct-q4_K_M
Gemma 3 12B InstructGemma12BQ4_K_M16 GB~9.5 GBYesgemma3:12b
Gemma 4 12BGemma12BQ4_K_M12 GB~8 GBYesgemma4:12b
Gemma 4 12B (Q8)Gemma12BQ8_024 GB~12.8 GBYesgemma4:12b-it-q8_0
Gemma 4 26B-A4BGemma26BQ4_K_M24 GB~16 GBYesgemma4:26b
Gemma 4 26B-A4B (Q8)Gemma26BQ8_048 GB~28.1 GBYesgemma4:26b-a4b-it-q8_0
Gemma 2 27B InstructGemma27BQ4_K_M32 GB~21 GBYesgemma2:27b-instruct-q4_K_M
Gemma 3 27B InstructGemma27BQ4_K_M32 GB~21 GBYesgemma3:27b
Gemma 4 31BGemma31BQ4_K_M32 GB~20 GBYesgemma4:31b
Gemma 4 31B (Q8)Gemma31BQ8_048 GB~30.9 GBYesgemma4:31b-it-q8_0
Gemini 2.5 ProGoogleUndisclosedAPI0 GBn/aCloudn/a
Gemini 2.5 FlashGoogleUndisclosedAPI0 GBn/aCloudn/a
Gemini 3.1 ProGoogleUndisclosedAPI0 GBn/aCloudn/a
GPT-OSS 20BGPT-OSS21BMXFP424 GB~13.8 GBYesgpt-oss:20b
GPT-OSS 120BGPT-OSS117BMXFP496 GB~65.4 GBYesgpt-oss:120b
Granite 4.1 3B InstructGranite3BQ4_K_M4 GB~2 GBYesgranite4.1:3b
Granite 4.2 3BGranite3.7BQ4_K_M4 GB~2.1 GBYesgranite4.2:3b
Granite 4.1 8B InstructGranite8BQ4_K_M8 GB~5.5 GBYesgranite4.1:8b
Granite 4.2 8BGranite8.8BQ4_K_M8 GB~5 GBYesgranite4.2:8b
Granite Switch 4.1 8BGranite8.8BQ4_K_M12 GB~6.4 GBYesn/a
Granite 4.2 30BGranite29.3BQ4_K_M24 GB~16.5 GBYesgranite4.2:30b
Granite Switch 4.1 30BGranite30BQ4_K_M32 GB~21 GBYesn/a
Inkling SmallInkling276BAPI0 GBn/aOpen (llama.cpp)n/a
Kimi K2 InstructKimi1000BAPI0 GBn/aOpen, no fitn/a
Kimi K2.6Kimi1000BAPI0 GBn/aOpen, no fitn/a
Kimi K2.7-CodeKimi1000BAPI0 GBn/aOpen, no fitn/a
Kimi K3Kimi2800BAPI0 GBn/aOpen, no fitn/a
Kolibri 1Kolibri78.1BQ4_K_M96 GB~49.8 GBYesn/a
Kolibri 1 (IQ4_XS)Kolibri78.1BIQ4_XS64 GB~43.6 GBYesn/a
Poolside Laguna XS.2Laguna33BQ4_K_M36 GB~23 GBYeslaguna-xs.2:q4_K_M
Laguna XS 2.1Laguna33BQ4_K_M32 GB~20.3 GBYeslaguna-xs-2.1:q4_K_M
Laguna S 2.1Laguna118BQ4_K_M128 GB~96 GBYeslaguna-s-2.1:q4_K_M
LFM2.5 8B-A1BLFM28.3BQ4_K_M8 GB~5.5 GBYeslfm2.5:8b-a1b-q4_K_M
LFM2 24B-A2B InstructLFM224BQ4_K_M24 GB~14 GBYeslfm2:24b-a2b
Ling-3.0-flashLing124BQ4_K_M128 GB~77.8 GBYesn/a
Ling-3.0-flash (IQ4_XS)Ling124BIQ4_XS96 GB~68.7 GBYesn/a
Llama 3.2 1B InstructLlama1BQ4_K_M2 GB~1 GBYesllama3.2:1b-instruct-q4_K_M
Llama 3.2 3B InstructLlama3BQ4_K_M4 GB~2.5 GBYesllama3.2:3b-instruct-q4_K_M
Llama 3.1 8B InstructLlama8BQ4_K_M12 GB~6.5 GBYesllama3.1:8b-instruct-q4_K_M
Llama 3.1 8B Instruct (Q8)Llama8BQ8_016 GB~8 GBYesllama3.1:8b-instruct-q8_0
Llama 3.1 8B Instruct (Q5)Llama8BQ5_K_M12 GB~8 GBYesllama3.1:8b-instruct-q5_K_M
Llama 3.1 70B InstructLlama70BQ4_K_M64 GB~42 GBYesllama3.1:70b-instruct-q4_K_M
Llama 3.3 70B InstructLlama70BQ4_K_M64 GB~42 GBYesllama3.3:70b-instruct-q4_K_M
Llama 3.3 70B Instruct (Q8)Llama70BQ8_096 GB~75 GBYesllama3.3:70b-instruct-q8_0
Llama 3.3 70B Instruct (Q6)Llama70BQ6_K96 GB~57.9 GBYesllama3.3:70b-instruct-q6_K
Llama 4 ScoutLlama109BQ4_K_M96 GB~67 GBYesllama4:scout
Llama 4 MaverickLlama400BQ4_K_M320 GB~245 GBYesllama4:maverick
Llama 3.1 405B InstructLlama405BQ4_K_M320 GB~243 GBYesllama3.1:405b-instruct-q4_K_M
Mellum2.1 12B-A2.5B ThinkingMellum12.1BQ4_K_M16 GB~8.1 GBYesn/a
MiMo-V2.6-Distill-Qwen-9BMiMo9BQ4_K_M16 GB~5.8 GBYesn/a
Xiaomi MiMo-V2-FlashMiMo309BAPI0 GBn/aOpen (llama.cpp)n/a
Xiaomi MiMo-V2.6-FlashMiMo309BAPI0 GBn/aOpen (llama.cpp)n/a
Xiaomi MiMo-V2.6-ProMiMo1020BAPI0 GBn/aOpen, no fitn/a
MiniCPM-V 4.5 8BMiniCPM8.7BQ4_K_M12 GB~5.7 GBYesminicpm-v4.5:8b
MiniMax M3MiniMax428BAPI0 GBn/aOpen, no fitn/a
MiniMax M3.1 FlashMiniMax428BAPI0 GBn/aOpen, no fitn/a
Ministral 3 3BMistral3.4BQ4_K_M6 GB~3 GBYesministral-3:3b
Mistral 7B InstructMistral7BQ4_K_M8 GB~5.5 GBYesmistral:7b-instruct-v0.3-q4_K_M
Mistral 7B Instruct (Q5)Mistral7BQ5_K_M8 GB~4.8 GBYesmistral:7b-instruct-v0.3-q5_K_M
Mistral 7B Instruct (Q8)Mistral7BQ8_016 GB~7.2 GBYesmistral:7b-instruct-v0.3-q8_0
Ministral 3 8BMistral8.4BQ4_K_M12 GB~6 GBYesministral-3:8b
Mistral Nemo 12BMistral12BQ4_K_M16 GB~9.5 GBYesmistral-nemo:12b
Mistral Nemo 12B (Q8)Mistral12BQ8_024 GB~12.1 GBYesmistral-nemo:12b-instruct-2407-q8_0
Ministral 3 14BMistral13.5BQ4_K_M16 GB~9.1 GBYesministral-3:14b
Mistral Small 22BMistral22BQ4_K_M32 GB~17 GBYesmistral-small:22b
Mistral Small 3.1Mistral24BQ4_K_M24 GB~15 GBYesmistral-small3.1:24b
Mistral Small 3.1 (Q8)Mistral24BQ8_036 GB~23.3 GBYesmistral-small3.1:24b-instruct-2503-q8_0
Devstral Small 2 24BMistral24BQ4_K_M24 GB~15.2 GBYesdevstral-small-2:24b
Mixtral 8x7B InstructMistral46.7BQ4_K_M48 GB~30 GBYesmixtral:8x7b
Devstral 2 123BMistral123BQ4_K_M96 GB~74.9 GBYesdevstral-2:123b
Mistral Medium 3.5Mistral128BAPI0 GBn/aOpen (llama.cpp)n/a
Muse Glimmer 30BMuse29.8BQ4_K_M32 GB~16.9 GBYesmuse-glimmer:30b
Muse Spark OpenMuse816BAPI0 GBn/aOpen, no fitn/a
NVIDIA Nemotron 3.5 Bolt E4BNemotron4BQ4_K_M12 GB~7.9 GBYesnemotron-3.5-bolt:e4b
NVIDIA Nemotron Cascade 2 30B-A3BNemotron30BQ6_K36 GB~24 GBYesnemotron-cascade-2:30b
Nemotron 3.5 Lightning 30B-A3BNemotron30BQ4_K_M36 GB~25.4 GBYesnemotron-3.5-lightning:30b
Nemotron 3.5 Lightning 30B-A3B (IQ4_XS)Nemotron30BIQ4_XS32 GB~18.9 GBYesn/a
Nemotron 3 Super 120B-A12BNemotron120BQ4_K_M128 GB~86.8 GBYesnemotron-3-super:120b
Nemotron 3 Super 120B-A12B (IQ4_XS)Nemotron120BIQ4_XS96 GB~64.5 GBYesn/a
NVIDIA Nemotron 3 UltraNemotron550BAPI0 GBn/aOpen, no fitn/a
Cohere North Mini CodeNorth30BQ4_K_M32 GB~19 GBYesnorth-mini-code-1.0:q4_K_M
Olmo 3.1 32B InstructOlmo32BQ4_K_M32 GB~19.5 GBYesolmo-3.1:32b
GPT-4oOpenAIUndisclosedAPI0 GBn/aCloudn/a
GPT-4o miniOpenAIUndisclosedAPI0 GBn/aCloudn/a
GPT-5.5OpenAIUndisclosedAPI0 GBn/aCloudn/a
Ornith 1.0 9BOrnith9BQ4_K_M8 GB~5.6 GBYesornith:9b
Ornith 1.0 35BOrnith35BQ4_K_M32 GB~21.2 GBYesornith:35b
Phi-3 Mini 3.8BPhi3.8BQ4_K_M6 GB~3.2 GBYesphi3:3.8b-mini-4k-instruct-q4_K_M
Phi-4 Mini 3.8BPhi3.8BQ4_K_M6 GB~3.2 GBYesphi4-mini:3.8b
Phi-4 Mini 3.8B (Q8)Phi3.8BQ8_08 GB~3.8 GBYesphi4-mini:3.8b-q8_0
Phi-3 Medium 14BPhi14BQ4_K_M16 GB~11 GBYesphi3:14b-medium-4k-instruct-q4_K_M
Phi-4 14BPhi14BQ4_K_M24 GB~11.5 GBYesphi4:14b-q4_K_M
Phi-4 14B (Q8)Phi14BQ8_024 GB~14.5 GBYesphi4:14b-q8_0
Qwen3.7-PlusQwenUndisclosedAPI0 GBn/aCloudn/a
Qwen2.5 0.5B InstructQwen0.5BQ4_K_M2 GB~0.8 GBYesqwen2.5:0.5b-instruct-q4_K_M
Qwen3.5 0.8B InstructQwen0.8BQ4_K_M2 GB~0.8 GBYesqwen3.5:0.8b
Qwen2.5 1.5B InstructQwen1.5BQ4_K_M4 GB~1.5 GBYesqwen2.5:1.5b-instruct-q4_K_M
Qwen3.5 2B InstructQwen2BQ4_K_M4 GB~1.8 GBYesqwen3.5:2b
Qwen2.5 3B InstructQwen3BQ4_K_M4 GB~2.5 GBYesqwen2.5:3b-instruct-q4_K_M
Qwen3.5 4B InstructQwen4BQ4_K_M6 GB~3.5 GBYesqwen3.5:4b
Qwen3.5 4B Instruct (Q8)Qwen4BQ8_08 GB~4.3 GBYesqwen3.5:4b-q8_0
Qwen2.5 7B InstructQwen7BQ4_K_M8 GB~5.5 GBYesqwen2.5:7b-instruct-q4_K_M
Qwen2.5 Coder 7BQwen7BQ4_K_M8 GB~5.5 GBYesqwen2.5-coder:7b
Qwen3 8BQwen8BQ4_K_M12 GB~6.5 GBYesqwen3:8b-q4_K_M
Qwen3 8B (Q8)Qwen8BQ8_016 GB~8.1 GBYesqwen3:8b-q8_0
Qwen3.5 9B InstructQwen9BQ4_K_M12 GB~7 GBYesqwen3.5:9b
Qwen3.5 9B Instruct (Q8)Qwen9BQ8_016 GB~10.7 GBYesqwen3.5:9b-q8_0
Qwen2.5 14B InstructQwen14BQ4_K_M16 GB~11 GBYesqwen2.5:14b-instruct-q4_K_M
Qwen2.5 Coder 14BQwen14BQ4_K_M16 GB~11 GBYesqwen2.5-coder:14b
Qwen3 14BQwen14BQ4_K_M16 GB~11 GBYesqwen3:14b-q4_K_M
Qwen3 14B (Q8)Qwen14BQ8_024 GB~15.9 GBYesqwen3:14b-q8_0
Qwen3.5 27B InstructQwen27BQ4_K_M24 GB~16 GBYesqwen3.5:27b
Qwen3.5 27B Instruct (Q8)Qwen27BQ8_048 GB~27.1 GBYesqwen3.5:27b-q8_0
Qwen3.6 27BQwen27BQ4_K_M32 GB~18 GBYesqwen3.6:27b
Qwen3.8 27BQwen27BQ4_K_M24 GB~16.5 GBYesqwen3.8:27b
Qwen3.8 27B (Q8)Qwen27BQ8_048 GB~27.1 GBYesqwen3.8:27b-q8_0
Qwen3.6 27B (Q8)Qwen27BQ8_048 GB~30 GBYesqwen3.6:27b-q8_0
Qwen3 30BQwen30BQ4_K_M32 GB~22 GBYesqwen3:30b-a3b-q4_K_M
Qwen3 30B (Q8)Qwen30BQ8_048 GB~30.3 GBYesqwen3:30b-a3b-q8_0
Qwen3.5 35B-A3B InstructQwen35BQ4_K_M32 GB~20 GBYesqwen3.5:35b-a3b
Qwen3.6 35B-A3BQwen35BQ4_K_M32 GB~22 GBYesqwen3.6:35b-a3b
Qwen3.6 35B-A3B (Q8)Qwen35BQ8_064 GB~38.7 GBYesqwen3.6:35b-a3b-q8_0
Qwen3.5 35B-A3B Instruct (Q8)Qwen35BQ8_064 GB~38.7 GBYesqwen3.5:35b-a3b-q8_0
Qwen3-Next 80B-A3BQwen80BQ4_K_M72 GB~50.4 GBYesqwen3-next:80b
Qwen3-Next 80B-A3B (Q8)Qwen80BQ8_0128 GB~84.8 GBYesqwen3-next:80b-a3b-instruct-q8_0
Qwen3.5 122B-A10B InstructQwen122BQ4_K_M96 GB~72 GBYesqwen3.5:122b-a10b
Qwen3.8-Flash-NextQwen125BQ4_K_XL192 GB~111.3 GBYesn/a
Qwen3.8-Flash-Next (IQ4_XS)Qwen125BIQ4_XS128 GB~93.7 GBYesn/a
Qwen3.8-Flash-Next (Q3)Qwen125BQ3_K_XL128 GB~90 GBYesn/a
Qwen3 235B A22BQwen235BQ4_K_M192 GB~130 GBYesqwen3:235b-a22b-q4_K_M
Qwen3.8 2.4T A95BQwen2400BAPI0 GBn/aOpen, no fitn/a
SmolLM2 360MSmolLM0.36BQ4_K_M1 GB~0.5 GBYessmollm2:360m
Grok 4.3xAIUndisclosedAPI0 GBn/aCloudn/a
GLM-4 PlusZhipuUndisclosedAPI0 GBn/aCloudn/a
GLM-4.7-Flash 30B-A3BZhipu30BQ4_K_M32 GB~19 GBYesglm-4.7-flash
GLM-5.3 FlashZhipu321BAPI0 GBn/aOpen (llama.cpp)n/a
GLM-5Zhipu744BAPI0 GBn/aOpen, no fitn/a
GLM-5.1Zhipu744BAPI0 GBn/aOpen, no fitn/a
GLM-5.2Zhipu753BAPI0 GBn/aOpen, no fitn/a
GLM-5.3Zhipu753BAPI0 GBn/aOpen, no fitn/a

Estimated load = approximate memory at Q4_K_M; estimates, not measured. All local entries are GGUF builds pulled via Ollama, so they also run in llama.cpp and LM Studio. See the hardware stats for RAM-tier guidance.

Frequently asked questions

What does the ModelFit compatibility matrix show?

Every model ModelFit tracks (172 total, 129 local), with its parameter size, quantization, minimum RAM, and estimated memory load, so you can see at a glance which local AI models run on which Apple Silicon or NVIDIA hardware.

Can I download the dataset?

Yes. A machine-readable JSON export is free at /api/dataset/, licensed CC BY 4.0. The same data is also mirrored on GitHub and Hugging Face for offline use or bulk analysis.

Does "runs locally: false" mean a model is closed?

No. runsLocally tracks whether a registry-verified Ollama build fits a consumer RAM tier ModelFit maps (up to 256GB). Some open-weight models publish weights but exceed every tier. NVIDIA Nemotron 3 Ultra (550B, OpenMDW license) needs roughly 190 GB even at 2-bit quantization, so it is marked openWeights: true with runsLocally: false.

How is model fit calculated?

A model needs roughly 0.6 GB of memory per billion parameters at Q4_K_M quantization. ModelFit sizes recommendations to ~70% of a device's unified memory up to 32GB, scaling to ~85% at 128GB and above, leaving headroom for the OS, context, and KV-cache. On high-RAM Macs you can raise the GPU-wired ceiling further with iogpu.wired_limit_mb.