IBM Granite Switch 4.1 GGUF on a Mac: 8B Fits 16GB, 30B Needs 32GB

IBM Granite Switch 4.1 now runs on a Mac from IBM's own GGUF files, and a 16GB Mac holds the 8B. Its Q4_K_M file is 6,412,400,416 bytes, and the 30B Q4_K_M is 21,035,547,008 bytes, which fits a 32GB Mac with little room left for context (IBM 8B GGUF, 2026). Each file carries the chat model plus two embedded adapters, Decision and Guardian. It does not carry the speech-to-text encoder that IBM's launch post advertises, so on a Mac you get chat and decisions, not audio.

TL;DR: 16GB Mac: the 8B at Q4_K_M, with about 4.8 GB of budget left for context. 32GB Mac: the 30B at Q4_K_M fits with about 1.4 GB left, enough for roughly 5,000 tokens; 48GB is the comfortable size. Run it with llama.cpp build b11798 or newer. There is no Ollama library tag, and no GGUF includes the speech encoder.

What is Granite Switch 4.1?

Granite Switch 4.1 is IBM's Granite 4.1 base model with small adapters composed into one checkpoint, so one server can answer chat requests and structured decision questions. The 8B model card lists the base as ibm-granite/granite-4.1-8b, two aLoRA adapters (decision and guardian-core), and an embedded English speech encoder, Granite Speech 5.0 TurboCTC, with 470M parameters. The release date is October 14, 2026, under Apache 2.0.

The adapters switch on through control tokens in the chat template. Decision answers a multiple-choice question about a state with a probability for each option. Guardian Core flags safety risks such as harm, jailbreaking and violence. IBM's BUILD.md puts the composed 8B at 8,776,400,896 parameters, 4.72% more than the 8,380,551,168-parameter base. The 30B grows from 28,865,728,512 to 29,989,814,272 parameters (IBM 30B model card, 2026).

Both sizes are dense models with a 131,072-token context, the same as plain Granite 4.1. We covered the plain 3B and 8B in should you run IBM Granite 4.1 on your Mac. This post is only about the Switch builds and what their GGUF files contain.

What does one Granite Switch GGUF actually contain?

One file holds the base weights, both adapters and the adapter-aware chat template. It does not hold the speech encoder. That answer comes from the GGUF header of the Q4_K_M files, which ModelFit read directly, because the GGUF repository README is a one-paragraph pointer to the base model card.

The 8B header declares the graniteswitch architecture, 40 blocks, adapters.count = 2, and a chat template that maps guardian-core and decision to their control tokens. Its 842 tensors are the transformer weights plus LoRA A and B matrices for every block. No tensor belongs to an audio encoder, and the repository ships no separate projector file. The 30B header is the same design with 64 blocks.

The base safetensors repository does not include the speech weights either. Its weight index lists only the language model, the LoRA slices and two switch tables. IBM's card describes speech as a cascade that transcribes audio and splices the text into the chat, and it lists Transformers and vLLM as the backends. So treat speech as a Linux and vLLM feature for now, not something a Mac gets from these files.

Does Granite Switch run on a Mac today?

Yes, through llama.cpp, which uses Metal on Apple Silicon. Mainline llama.cpp merged the Granite-Switch architecture in pull request 25107 on August 10, 2026, and Decision support for the 8B in pull request 30676 on October 16, 2026.

IBM's conversion metadata names the exact build. Each repository has a -bf16-metadata.properties file reading llamacpp_build_tag=b11798, a release published on October 21, 2026, with a macOS arm64 binary. The llama.cpp server README at that build says it plainly: "A decision model that is also a text model, such as granite-switch, serves both." It caps a Granite Switch choice question at 255 options (llama.cpp server README, 2026).

Three gaps are worth knowing before you install:

• No Ollama tag. The registry returns 404 for granite-switch today, so there is no ollama run command to give.
• No IBM MLX build. Two community MLX conversions appeared on October 23, 2026, but IBM publishes none, and ModelFit has not checked them.
• Guardian on llama.cpp is undocumented. The adapter sits in the file and in the template. IBM's Guardian examples target vLLM, and the llama.cpp README names only the Decision path. Assume it may work through chat_template_kwargs, but nobody has documented it.

How much memory does the 8B need on a Mac?

The 8B needs a 16GB Mac at Q4_K_M or Q6_K, and an 8GB Mac only at a quality-damaging Q2_K or Q3_K_S. Sizes below are byte totals from the Hugging Face file listing of the 8B GGUF repository, read on October 25, 2026, in decimal megabytes.

The budget column applies ModelFit's sizing rule, the one behind every pick in the homepage wizard. The engine allows 70% of unified memory at 32GB and below, ramping to 85% at 128GB and above, because macOS limits how much memory the GPU can wire. That is 5,600 MB on 8GB, 11,200 MB on 16GB and 22,400 MB on 32GB. The full ladder is in the RAM guide for local LLMs.

How much memory does the 30B need?

The 30B fits a 32GB Mac only at Q4_K_M and below, and Q4_K_M leaves 1,364 MB of the 22,400 MB budget. A 48GB Mac, with a 34,800 MB budget, is the size that runs it with real context. Sizes come from the 30B GGUF repository file listing.

So the 30B is a 32GB model for the file, not for long chats. A 24GB Mac gets only Q2_K, which loses enough quality that the 8B at Q6_K is the better choice there.

Why is the Switch GGUF bigger than plain Granite 4.1?

Because the adapters ride along at full precision. In the 8B Q4_K_M file, the LoRA tensors are stored in BF16 and take about 1,188 MB, while the quantized base weights take about 5,221 MB. ModelFit summed those from the tensor offsets in the file header. For the 30B the adapter share is about 3,372 MB.

For comparison, Ollama's plain granite4.1:8b tag has a 5,347,914,400-byte weight layer. The Switch file is about 1,064 MB larger for the same 8B base, and nearly all of that is the two adapters. You pay that memory even if you only ever chat. If you never need Decision or Guardian, plain Granite 4.1 8B is the leaner download.

How much context fits?

Far less than the 131,072-token maximum on a base-size Mac. The KV cache, at llama.cpp's default 16-bit cache type, costs 163,840 bytes per token on the 8B and 262,144 bytes per token on the 30B. That is ModelFit's arithmetic from the GGUF header: 8 key-value heads of 128 dimensions, keys and values, times 40 or 64 blocks.

On a 16GB Mac, the 4,788 MB left after the 8B Q4_K_M holds about 29,000 tokens. On a 32GB Mac, the 1,364 MB left after the 30B Q4_K_M holds about 5,200 tokens. The full context would need about 21 GB of cache on the 8B and 34 GB on the 30B, before the weights. Decision questions use short prompts, so they fit easily. Long document chats on the 30B want 48GB or more.

How fast will Granite Switch be on a Mac?

No published Mac speed exists, so the figures below come from ModelFit's engine, which now carries a row for each Switch file. It reads the 6,412,400,416-byte and 21,035,547,008-byte Q4_K_M files and applies the speed model behind every pick in the homepage wizard. Because it counts the adapter weights as memory read on every token, the rows land below plain Granite 4.1 8B and Granite 4.2 30B.

These are engine estimates at a 4,096-token context, not measurements. Both Switch files are larger than plain Granite: the 8B by about 1.1 GB and the 30B by about 3.3 GB against the 17,721 MB Ollama granite4.2:30b layer, which is why the table sits below the plain models' figures. To see what else your exact Mac runs, the free CLI npx @wecko-ai/modelfit prints the engine's ranked picks.

How do you run Granite Switch on a Mac?

Install a llama.cpp build at b11798 or newer, then point the server at IBM's repository. The -hf flag downloads the file and defaults to Q4_K_M when you give no quant.

llama-server -hf ibm-granite/granite-switch-4.1-8b-GGUF:Q4_K_M --port 8080
# 30B on a 48GB Mac
llama-server -hf ibm-granite/granite-switch-4.1-30b-GGUF:Q4_K_M --port 8080

Chat goes to /v1/chat/completions as with any llama.cpp model. Decisions go to /v1/systemone, which takes a state and typed questions (choice, score or noul) and returns a probability for each option. The server reports the model as ["text", "decisions"], so a client can find both paths without probing.

What the Decision adapter is worth depends on your task. On IBM's Decision Index fast subset, the 8B scores 42.9 overall and the 30B scores 50.1, with Tools at 59.8 and 76.5. Those are IBM's figures from a hosted vLLM endpoint (IBM 8B model card, 2026), not Mac runs. IBM also asks you to treat the probabilities as a prior, not a certified classification.

FAQ

Can a 16GB Mac run Granite Switch 4.1?

Yes, the 8B. Its Q4_K_M GGUF is 6,412 MB, and a 16GB Mac's engine budget is 11,200 MB, which leaves 4,788 MB for context, about 29,000 tokens. Q6_K at 8,067 MB also fits. The 30B does not fit a 16GB Mac at any quant IBM publishes.

Can a 32GB Mac run the 30B?

Yes, at Q4_K_M or Q3_K_M. The Q4_K_M file is 21,036 MB against a 22,400 MB budget, so only about 5,200 tokens of context fit. Q3_K_M leaves 4,888 MB at lower quality. For long chats with the 30B, a 48GB Mac is the realistic size.

Does the GGUF include speech-to-text?

No. The GGUF header lists the language model and the two adapters only, and the repositories ship no audio projector. IBM's speech cascade uses a separate model, Granite Speech 5.0 TurboCTC, and IBM documents it on Transformers and vLLM. On a Mac you get chat and decisions from these files.

Is there an Ollama command for Granite Switch?

Not today. The Ollama registry has no granite-switch library tag, so ModelFit does not publish an ollama run line for it. Use llama.cpp's llama-server with the -hf flag, build b11798 or newer, which serves chat and the /v1/systemone decision API from the same process.

Should I pick Granite Switch or plain Granite 4.1?

Pick Switch if you want calibrated multiple-choice decisions or a safety check from the model you already chat with. Pick plain Granite 4.1 if you only chat: the 8B Switch file carries about 1.2 GB of adapter weights you would never use. The ModelFit benchmark compares the plain models.

Sources

• ibm-granite/granite-switch-4.1-8b-GGUF and its file listing (IBM, 2026)
• ibm-granite/granite-switch-4.1-30b-GGUF and its file listing (IBM, 2026)
• Granite Switch 4.1 8B model card and BUILD.md (IBM, October 14, 2026)
• Granite Switch 4.1 30B model card and BUILD.md (IBM, October 14, 2026)
• Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint (IBM on Hugging Face, October 15, 2026)
• llama.cpp pull request 25107, Granite-Switch architecture and pull request 30676, Granite switch decision (GitHub, 2026)
• llama.cpp build b11798 and its server README (GitHub, October 21, 2026)
• Ollama library: granite4.1 (Ollama, 2026)