Ling-3.0-flash-VL Puts 124B Parameters in a Model That Computes Only 5.5B: What a Mac Needs

Ling-3.0-flash-VL is InclusionAI's open-weight vision-language model, and it is the cleanest sparse mixture-of-experts test case of 2026 for Apple Silicon. A mixture-of-experts (MoE) model is one that stores far more parameters than it computes per token, and that gap is the whole story here. The file holds 124B parameters in memory, while each token touches only 5.5B of them. Consequently, you pay large-model memory for small-model compute, and that split decides everything on a Mac.

So the question stops being whether your machine can run a 124B model. Instead, it becomes which quantization tier survives after macOS takes its share of unified memory. We answer with the published numbers, and we checked every file size in this article against the Hugging Face API rather than trusting a press release.

Does Ling-3.0-flash-VL run on your Mac? The short answer

Yes, from 48GB of unified memory upward, and only through the GGUF route. GGUF is the llama.cpp container format that packs quantized weights, tensor metadata and the tokenizer into a single file, and it is the only format that carries these quants today. The floor is arithmetic, not vibes.

The most popular file, bartowski's Q4_K_M, is 78.66 GB. For example, a 96GB Mac leaves roughly 77GB usable once you count macOS and system services, so Q4_K_M does not fit there. So which file should a 96GB owner load? IQ4_XS at 69.37 GB or Q4_K_S at 73.98 GB, because both fit where the popular pick does not.

The rest of the ladder follows the same pattern, and the tier list is tighter than the parameter count suggests:

• 36GB Mac: IQ1_S at 28.07 GB, marginal, with almost no room for context.
• 48GB Mac: IQ1_M at 31.30 GB comfortably, IQ2_XXS at 36.69 GB if you accept a tight fit.
• 64GB Mac: IQ2_M at 45.32 GB, or Q2_K at 47.81 GB on the edge.
• 96GB Mac: IQ4_XS at 69.37 GB, or Q4_K_S at 73.98 GB tight. Q4_K_M does not fit.
• 128GB Mac: Q4_K_M at 78.66 GB with room to spare, up to Q5_K_S at 88.96 GB.
• 192GB and up: Q8_0 at 132.37 GB fits, and a 512GB machine loads the full bf16 file at 248.94 GB.

Context is cheap here, which is unusual. At 128K tokens the whole conversation cache costs about 1.9 GiB. As a result, weights decide the memory budget and almost nothing else does. There is also no one-line Ollama tag for this model; we checked the library on September 27, 2026 and every Ling variant returned 404. The working path is llama.cpp instead, which merged Ling 3.0 VL support on September 24, 2026, so you need release b11159 or newer.

What Ling-3.0-flash-VL actually is

InclusionAI published the weights under the MIT licence, so you can ship them commercially without a revenue gate. That makes this release simpler to ship than the revenue-gated finetunes that dominated September 2026. It is also a native multimodal model, not a text model with a bolted-on vision adapter.

Two numbers deserve a caution. Bartowski's model card lists the parameter count as 125B while InclusionAI says 124B, and the GGUF metadata resolves to 124.4B. Therefore the headline 124B is the vendor's own rounding. The Artificial Analysis score of 42 is also a vendor measurement. We have not reproduced it, and no one has published an independent run of the Intelligence Index on this checkpoint.

On context, the released configuration is the honest number. config.json sets max_position_embeddings to 131072, which is 128K, and the GGUF metadata carries the same value. YaRN scaling gets you to the 256K figure in the model card. In practice, InclusionAI's own serving recipe uses --context-length 262144 with YaRN enabled on a four-GPU node.

Why the sparse MoE math works in your favour

A mixture-of-experts model does not shrink your memory requirement. Every one of the 124B parameters has to sit somewhere the machine can reach, so the total sets capacity. What sparsity changes is the read pattern per token, and that read pattern sets decode speed.

The arithmetic is short. Bartowski's Q4_K_M file is 78.66 GB for 124.4B parameters, which works out to about 0.63 GB per billion, or roughly 5.1 bits per weight. That matches the 0.6 GB per billion rule of thumb we use in our VRAM cheat sheet, so it is a useful sanity check on the file itself.

Now multiply by the active set instead of the total. At 5.5B active parameters and 0.63 GB per billion, a single token reads about 3.48 GB of weights rather than 78.66 GB. That is a 23x reduction in bytes moved per token, and it is the reason a model this size can feel responsive at all. Take the same file with a dense architecture of the same size and a token would read all 78.66 GB; here it reads 3.48 GB. The same logic made the streaming experiments in our 744B honest-math breakdown worth writing about. Here, however, nothing has to come off a disk.

Where the context cache goes

This is where the hybrid backbone earns its keep, and it is the most interesting part of the model for a Mac buyer. A KV cache is the per-token attention state a model keeps for the text it has already read, and in most models it grows linearly with the conversation. Here it barely moves.

Of the 42 layers, only 7 are Gated MLA layers that keep a key-value cache which grows with context. The other 35 are KDA layers, a linear-attention variant that keeps a fixed-size recurrent state and adds nothing as the conversation lengthens. The configuration's layer_group_size of 6 over 42 layers produces exactly that 7 and 35 split. That split is our arithmetic from the released config rather than a vendor claim.

Per MLA layer, the cache stores a compressed latent of 512 values plus a decoupled rotary key of 64 values, so 576 on the key side and 512 on the value side, 1,088 values in total. At f16 that is 2,176 bytes per token per layer, or 15,232 bytes per token across the seven MLA layers. Call it 14.9 KiB per token.

Compare that with an ordinary attention model of the same shape. If all 42 layers stored full key-value pairs at 32 heads and 128 head dimensions, the cache would run 672 KiB per token. That is 21 GiB at 32K and 84 GiB at 128K. So the hybrid design is roughly 45 times cheaper per token. Consequently, weights limit long-context Ling sessions rather than cache. Our memory calculator applies the same per-token method to other hybrid models.

Mac compatibility by memory tier

These verdicts add up published file sizes, our per-token cache figure, and the 1 to 2 GB allowance our device guide applies for the compute buffer and the vision projector. Neither a vendor nor a community has published a measurement of these files on Apple Silicon, and we did not run one. So read them as arithmetic.

Two Macs deserve a specific note. The M5 Max with 96GB runs its 614 GB/s bus across the whole line, so the 96GB tier does not add speed over 64GB, only capacity. The 96GB M5 Ultra doubles that to roughly 1.2 TB/s, which matters for the decode ceiling below even though it lands at the same usable memory. On both, capacity decides which file you can load. Which tier you should load is a separate question, and it is a question of trust at 4 bits for a multimodal agent.

There is no published KLD or quality data for these tiers. Nobody has measured how much the IQ1 and IQ2 builds lose on this specific model. Take the 36GB Mac as the sharpest example: the only file that fits there is IQ1_S, which is also the file with the least published quality evidence behind it. As a result, choosing the lightest file is a bet rather than a measured trade.

Does memory bandwidth bottleneck the 5.5B active path?

Partly, and the arithmetic ceiling is easy to state. Memory bandwidth is the rate at which a chip moves bytes from unified memory into the compute units, and on Apple Silicon it is the hard floor under decode speed.

At 3.48 GB of weights read per token, a 614 GB/s bus puts the ceiling near 177 tokens per second, and a 1.2 TB/s bus near 345. Those are arithmetic limits, not measurements. Real decode also pays for attention over the cache, router work across 512 experts, and any memory-controller inefficiency. Therefore actual throughput sits well under the ceiling. So how should a Mac buyer read the 177 figure? As a ceiling, not as a promise. Frame it as this: bandwidth is the wall the active path runs into, and it is a much further wall than a dense model of the same total size would face.

If InclusionAI's 5.1B active figure from the llama.cpp pull request is the true one, the read per token drops to about 3.22 GB and the ceilings rise to roughly 190 and 372 tokens per second. Either way, the ranking between machines does not change. Neither does the conclusion that sparsity moves the bottleneck rather than removing it.

What we cannot tell you is the real number. Neither InclusionAI nor bartowski publishes a single tokens-per-second figure for these files on Apple Silicon, and we have not measured one. Any Mac speed claim you read for this model is extrapolated, and you should treat it as such.

What you need to install

The runtime question is settled. llama.cpp merged support for the vision-language variant in pull request #29151 on September 24, 2026, and it shipped in release b11159 the same day. That change adds the 27-block vision tower, its merger and the two-layer projector on top of the bailingmoe3 text backbone, and it registers the LING3VL projector type. The pull request author reports end-to-end image and video inference working on llama-server. In other words, the GGUF path is not text only.

llama-server -hf bartowski/Ling-3.0-flash-VL-GGUF:Q4_K_M

That single command pulls the weights and the matching projector file automatically. If you load files by hand, pass the projector with --mmproj. The two projector files in the repository are mmproj-Ling-3.0-flash-VL-bf16.gguf at 0.88 GB and mmproj-Ling-3.0-flash-VL-f16.gguf at 0.87 GB, and they pair with any quant.

One warning on versions. Bartowski built these quants with b11159 and states plainly that the architecture needs that release or newer. On an older build the file will not load at all, so check the build tag before you blame the download.

What we could not verify

• Any independent reproduction of the Artificial Analysis score of 42. It is InclusionAI's own measurement, paired with its own evaluation protocol.
• The exact KV cache llama.cpp allocates at runtime for this hybrid. Our 14.9 KiB per token is derived from the config's MLA widths and the 7-of-42 layer split. If the build stores only the 576-wide compressed latent rather than the full 1,088 values, the figure halves to about 7.9 KiB per token.
• Decode speed on Apple Silicon for any of these files. Nothing is published.
• Quality loss for the IQ1, IQ2 and IQ3 tiers on this model. No KLD table exists for it.
• Whether the 256K YaRN context is usable in llama.cpp at all. The GGUF metadata carries 128K, and InclusionAI validated YaRN beyond that in its four-GPU server recipe, not on a Mac.
• Whether LM Studio, koboldcpp or Jan load the vision projector correctly on their current bundled runtimes. The repository lists them as compatible and we tested none.

FAQ

Can a 96GB Mac run Ling-3.0-flash-VL?

Yes, but not at Q4_K_M. The Q4_K_M file is 78.66 GB and macOS leaves roughly 77GB usable on a 96GB machine. Use IQ4_XS at 69.37 GB or Q4_K_S at 73.98 GB instead, and keep the context moderate. A 96GB Mac Studio or M5 Ultra both work at those tiers.

Is Ling-3.0-flash-VL free to use commercially?

Yes. InclusionAI released the weights under the MIT licence, which permits commercial use without a revenue gate or a separate enterprise agreement. That is a simpler position than the PolyForm and revenue-capped licences used by several September 2026 finetunes.

How much memory does the context window cost?

Very little by the standards of a 124B model. The hybrid backbone caches key-value pairs on only 7 of its 42 layers, so the cache runs about 14.9 KiB per token at f16. That is 0.47 GiB at 32K and 1.86 GiB at 128K. Therefore the weights dominate the budget, not the conversation.

Does it need a special llama.cpp build?

Yes. Support for the vision-language variant and its projector merged on September 24, 2026 and landed in release b11159. Bartowski built his quants with b11159, and the repository states that this release or newer is required. Older builds refuse the file.

Does the multimodal part work in GGUF, or is it text only?

Both image and video input work in llama.cpp. The repository ships two projector files, and the pull request author reports end-to-end image and video inference succeeding on llama-server. Automatic download of the projector happens when you use the -hf flag, or you pass --mmproj manually.

Sources

• inclusionAI/Ling-3.0-flash-VL model card and its config.json, read September 27, 2026
• Hugging Face API metadata for the licence, architecture tag and download counts
• bartowski/Ling-3.0-flash-VL-GGUF for the quantization table, file sizes and projector files, read September 27, 2026
• llama.cpp pull request #29151, model: add Ling 3.0 VL support, merged September 24, 2026
• llama.cpp release b11159, September 24, 2026, and src/models/bailingmoe3.cpp with tools/mtmd/models/ling3vl.cpp on master
• Download and file-size figures from the Hugging Face API for bartowski/Ling-3.0-flash-VL-GGUF and bloomer010/Ling-3.0-flash-VL-GGUF, September 27, 2026
• Peak bandwidth figures for the M5 Max and M5 Ultra from our own device guides, linked in the tier section above

Every number in this article is either a vendor figure we opened ourselves or our own arithmetic, labelled as such. How we work: about ModelFit. Corrections welcome: contact.