Yes, a 744B-parameter model can run on a machine with 25GB of RAM. A new open-source engine called colibri does it by streaming GLM-5.2's experts from an NVMe drive, and it earned over 1,200 GitHub stars in its first ten days (GitHub, 2026). The catch is speed: about 0.05 to 0.1 tokens per second on the author's reference machine, and 1.06 tokens per second even on an M5 Max with a 14 GB/s SSD. This post walks through why, because the math explains more about local AI hardware than most spec sheets do.
What is colibri, and what does it actually claim?
Colibri is a pure C inference engine with zero dependencies, released under Apache 2.0 on July 1, 2026 (GitHub, 2026). It runs GLM-5.2, the ~744B-parameter Mixture-of-Experts model whose weights Z.ai released under MIT (Hugging Face, 2026).
The trick rests on MoE sparsity. GLM-5.2 activates only about 40B of its 744B parameters per token, and per its config, each MoE layer routes to just 8 of 256 experts (config.json). Colibri splits the model in two:
- The dense part (attention, shared experts, embeddings, about 17B parameters) stays resident in RAM at int4: just 9.9GB.
- The 21,504 routed experts, about 19MB each at int4, live on disk in a ~370GB container and stream in on demand, backed by a per-layer LRU cache and the OS page cache (colibri README, 2026).
So the headline is true. The RAM requirement really is small. The question a hardware buyer should ask is what speed comes out the other end.
How fast does a disk-streamed 744B model actually run?
Between 0.05 and 1.06 tokens per second, depending on the machine. All figures below come from the colibri README and its linked GitHub issues; they are author-reported and community-reported numbers, not independent benchmarks.
| Machine | Disk | Reported speed |
|---|---|---|
| Reference dev box (12 cores, 25GB RAM, WSL2 VHDX) | ~1 GB/s random read | ~0.05 to 0.1 tok/s cold |
| Framework 13, Ryzen AI 9 HX 370, 128GB RAM | WD SN850X | 0.37 tok/s after cache tuning |
| Apple M5 Max, 128GB unified memory | internal SSD, 14.2 GB/s measured | 1.06 tok/s |
For scale: a chat reply of 200 tokens takes roughly 3 minutes at 1.06 tok/s and over half an hour at 0.1 tok/s. Colibri is a genuinely clever piece of engineering, and the author is upfront about these numbers. It is a research toy and a proof of concept, not a daily driver.
Why is a 7 GB/s SSD not fast enough?
Because expert streaming is a random-read workload, and SSD marketing numbers are sequential. A top PCIe 4.0 drive like the Samsung 990 Pro advertises 7,450 MB/s sequential reads (Samsung). Colibri's cold decode needs a different pattern: 75 MoE layers times 8 experts per token means roughly 11GB of reads per token, scattered across a 370GB file (colibri README, 2026).
Random reads across a file that large run far below the sequential spec. The author's dev box managed about 1 GB/s. Divide 1 GB/s by 11GB per token and you get the observed ~0.1 tok/s ceiling. The M5 Max datapoint is the strongest proof: its SSD sustained 14.2 GB/s in direct reads, yet output was 1.06 tok/s, because with a 23% expert cache hit rate most tokens still wait on thousands of small scattered reads.
This is the same lesson as our hardware bottleneck guide, one level down the memory hierarchy. RAM-resident models are bound by memory bandwidth. Disk-resident models are bound by random-read throughput. Neither is bound by the number on the box.
Does anything make it faster?
RAM does, which is the ironic part. The README's own analysis of the Framework 13 runs shows that on small-RAM machines the expert cache cap, not the disk, becomes the binding constraint: giving the cache more RAM lifted the hit rate from 28% to 66% and speed from 0.29 to 0.37 tok/s. The author's summary is worth quoting: "Every one of those constraints is a knob your machine can turn up" (colibri README, 2026).
In other words, the 25GB configuration proves the concept, and then the path to usable speed runs straight back through more RAM for caching hot experts. The hardware ladder does not disappear. It just moves.
Prior art points the same way. AirLLM streamed dense 70B layers through a 4GB GPU years earlier and landed under 1 tok/s (GitHub), and llama.cpp's mmap mode has always let the OS page weights in from disk. Colibri's advance is streaming only the 8 needed experts per layer instead of whole layers, which is why a 744B model is even in the conversation.
What should you run on your machine instead?
A model that fits in RAM with headroom will feel hundreds of times faster in practice. The general ladder:
| Your RAM | Comfortable model class |
|---|---|
| 8GB | 7B to 8B |
| 16GB | 13B to 14B |
| 24 to 32GB | 27B to 32B class, like Qwen3.6 27B |
| 64GB+ | 70B class or large MoE at Q4 |
On the quality side you give up less than the parameter counts suggest. Qwen3.6 27B scores 77.2 on SWE-Bench Verified and runs on a 24GB Mac (leaderboard). It generates in seconds, not minutes.
For your exact machine, the ModelFit wizard or a single npx @wecko-ai/modelfit in a terminal names the best pick for your hardware. The RAM sizing guide covers how much memory each class needs, and the context calculator shows how much of your budget long prompts consume. If you want the full compatibility matrix, it is open data on the data page.
FAQ
Can you really run GLM-5.2 on 25GB of RAM?
Yes. Colibri keeps GLM-5.2's ~17B dense parameters resident in 9.9GB of RAM at int4 and streams the 21,504 routed experts from a ~370GB file on NVMe. It works, and the author reports about 0.05 to 0.1 tokens per second cold on the 25GB reference machine. It is a proof of concept, not a practical assistant.
How fast is disk streaming compared to normal local inference?
Roughly 50 to 1,000 times slower. Colibri's best community-reported result is 1.06 tok/s on an M5 Max. A RAM-resident model of the class the same machine handles comfortably generates tens of tokens per second. The gap comes from reading about 11GB from disk per token in random access patterns.
Does a faster SSD fix the speed problem?
Only partially. Sequential ratings like 7,450 MB/s do not apply to the scattered small reads expert streaming produces. The M5 Max run sustained 14.2 GB/s in direct reads and still produced 1.06 tok/s. Past a point, the README's own data shows RAM for expert caching matters more than raw disk speed.
What does this mean if I am buying hardware for local AI?
The ladder still holds: RAM capacity decides which model class you run, and memory bandwidth decides how fast it feels. Disk streaming does not change the buying advice, it extends the same principle downward. Pick the model class your RAM fits comfortably; a fast NVMe is a nice-to-have, not a substitute for memory.
Sources
- Colibri repository and README: github.com/JustVugg/colibri
- GLM-5.2 weights and config: huggingface.co/zai-org/GLM-5.2
- Samsung 990 Pro specifications: samsung.com
- AirLLM layer streaming: github.com/lyogavin/airllm
- ModelFit local vs cloud leaderboard: modelfit.io/benchmark
Match this model to a machine that can run it: by RAM tier for Apple Silicon, or by VRAM for an NVIDIA GPU.
The weekly local-AI refresh
New open-weight models, real Apple Silicon benchmarks, and the one model worth running on your Mac this week. Free, one email a week, unsubscribe anytime.
By subscribing you agree to our Privacy Policy and to receive the weekly email. Unsubscribe anytime.
Have questions? Reach out on X/Twitter