Can a 512GB Mac Run Inkling 2 Small? 500GB Card Minimum

Short answer: the weights are small enough for a 512GB Mac Studio on paper, and no Mac engine is documented to load them. Thinking Machines opened the Inkling 2 Small weights on 12 November under Apache 2.0. The model card asks for at least 500GB of aggregated VRAM for the main checkpoint. The main Hub repo totals 393.5GB, the MXFP8 repo 622.8GB. A 256GB Mac cannot hold either. A 512GB Mac holds only the first, and the official engines are all GPU-cluster stacks.

Does a 512GB Mac Studio run Inkling 2 Small?

Not today, as far as anything public shows. Two separate questions hide in that sentence:

• Does it fit? The main NVFP4 repo is 393,510,045,676 bytes, which is 393.5GB in decimal units (Hub API). Apple lists 512GB of unified memory as a configuration of the Mac Studio with M5 Ultra (Apple tech specs). Weights alone use roughly 77% of that if Apple's GB equals a decimal GB, and roughly 72% if it means 2^30 bytes. Apple's page does not say which.
• Is there an engine? The model card lists SGLang, vLLM, TokenSpeed and Unsloth (Hugging Face model card). It names no MLX path and no llama.cpp path. On 13 November I found no MLX or GGUF repo for Inkling 2 Small on the Hub and no Inkling 2 support in mlx-lm or llama.cpp. Until one exists, "fits" is not "runs".

That is why this page gives no tokens-per-second figure for any Mac. A speed number without an engine would be invented.

What are the exact download sizes?

Thinking Machines announced Inkling 2 Small on 5 November, first behind its Tinker API. The weights went public on 12 November, per the update on the announcement page. This is a weights release, not a launch. Both repos were public and ungated when I read them on 13 November. Sizes below are decimal bytes from the Hub API, not binary GiB:

The card's example hardware for 500GB is 2x NVIDIA B300 or 4x NVIDIA H200. For 800GB it is 4x B300 or 8x H200 (model card). The model is a 594B-parameter mixture-of-experts with 17B active per token. It has 48 layers, routes 8 of 384 experts per token, takes a 1M-token context, and accepts text plus image input.

A mixture-of-experts (MoE) model is a network split into many expert sub-networks, of which a router picks only a few for each token. Every expert has to sit in memory even though only 17B are active per token, so the 594B figure sets the memory bill and the 17B figure only helps speed.

Two format names matter for the next section. NVFP4 is a 4-bit floating-point weight format from NVIDIA. MXFP8 is an 8-bit floating-point format from the microscaling (MX) family. The model card says Inkling 2 Small was trained in NVFP4, which is why the main repo uses it.

Which precision is the 393.5GB repo?

The main repo is the NVFP4 checkpoint, and the card recommends it for all use cases because the model was trained in NVFP4. The Hub API still tags the repo "8-bit", which looks like a contradiction. It is not. The repo is mixed precision, and the file list shows why:

• Expert weights, layers 0 to 39: NVFP4, stored as packed uint8. The Hub counts 247,669,456,896 U8 elements, about 248GB.
• Expert weights, layers 40 to 47: MXFP8, stored as FP8. The Hub counts 128,421,199,872 F8_E4M3 elements, about 128GB.
• Attention, embeddings, vision encoder, output head: bf16, 6,981,765,120 elements, about 14GB.

The layer assignment comes from hf_quant_config.json. The Hub's three tensor types account for about 390GB of the 393.5GB. I did not itemise the remainder. The FP8 tensors are what trigger the "8-bit" tag. A "394GB 4-bit model" is therefore the wrong label: the bulk is 4-bit, a real slice is 8-bit, and the rest is bf16.

Which Macs have enough memory?

Apple's page lists the Mac Studio with M5 Max at 36GB, configurable to 48GB, 64GB or 128GB, and with M5 Ultra at 96GB, configurable to 256GB or 512GB.

The MXFP8 row stays "No" under either unit convention: 622.8GB is still 73.1GB over a 512GB that counts 2^30 bytes (549.8 decimal GB). Clustering several Macs is outside this page, since no engine is documented for a single one.

What does a 512GB Mac have left after the weights?

Less than the headline suggests. The model card sets 500GB as its floor for a 393.5GB checkpoint, so Thinking Machines itself budgets about 106GB above the weights for cache and runtime. It does not break that number down. A 512GB Mac clears the card's 500GB by 12GB, and unified memory is shared with macOS, not dedicated GPU memory in a cluster. These are the items that eat the margin:

• macOS and apps. The system and your other processes come out of the same pool. macOS also limits how much unified memory the GPU may use by default; I did not verify the ceiling for this chip, and it can be raised, so check it before you size a build.
• KV cache at long context. From config.json: 40 of 48 layers use a sliding window of 1024 tokens, and the other 8 are global layers with 8 KV heads of dimension 128. A plain bf16 cache for those 8 layers at the full 1,048,576-token window comes to 32 GiB by my arithmetic. That is an estimate from the config, not a measurement. An engine may store more, since the config also lists a multi-token-prediction block and short convolutions, or less.
• Vision encoder activations. The encoder weights are already inside the repo totals, per the announcement, which gives the encoder at 292M parameters. Image inputs add runtime memory on top.

Read together: at short context, 393.5GB of weights inside 512GB looks workable on paper. At the full 1M window, the margin shrinks to a point where I would not promise it. Neither claim has been tested, because there is no engine to test with.

Can you run Inkling 2 Small on a Mac today?

I would not say so. What exists, as of 13 November:

• Documented engines: SGLang, vLLM, TokenSpeed and Unsloth, per the model card. Hosted APIs are listed too: NVIDIA, Fireworks, Together, Baseten, Nebius and Modal, per the announcement update.
• MLX: no Inkling 2 Small repo in mlx-community or elsewhere that I found. The mlx-community Inkling repos belong to the earlier Inkling and Inkling Small.
• GGUF and llama.cpp: no Inkling 2 Small GGUF found. The open llama.cpp pull request (#25731) adds the original Inkling architecture, is not merged, and I found no Inkling 2 reference in it. The new model declares its own architecture name (inkling2) in config.json, so do not assume the older work carries over.
• Unsloth: the model card lists it, but its Inkling page I read covers Inkling Small and the earlier Inkling, with no Inkling 2 Small section.

Check again before you buy or download anything. Support for a new architecture can land in days. When it does, the first question will be which quant, and a smaller quant would shrink the 393.5GB figure. No such quant exists yet, so no smaller size is claimed here.

What should you do, by Mac?

This is our call, derived from the sizes and the engine list above. It is a reading of public files, not a test result.

The pattern behind the table: the Mac question is decided by two independent gates. Unified memory is the single RAM pool that the CPU and GPU of an Apple Silicon Mac share, so the weights, the KV cache (the stored attention keys and values for the tokens already processed) and macOS all draw on it. Memory is the first gate, and only the 512GB tier clears it. The engine is the second gate, and nothing clears it today.

How is this different from Inkling Small?

Inkling Small is the earlier 276B-parameter model with 12B active, covered in Inkling Small on Mac Studio. It has MLX and GGUF builds that fit a 192GB Mac Studio. Inkling 2 Small is a different, larger model: 594B total, 17B active, a new architecture, and none of those builds yet. Do not reuse that article's sizes or commands.

For machines that can run big models today, see the best local LLMs for a 512GB Mac Studio, the 256GB list and the Mac Studio M5 Ultra 512GB guide. Another 512GB-class case is Tencent Hy4. For the general sizing rules, see how much VRAM you need for local LLMs.

FAQ

Can a 512GB Mac Studio run Inkling 2 Small?

The main repo, 393.5GB, fits in 512GB of unified memory on paper. No Mac engine (MLX or llama.cpp) is documented for it, so it does not run on a Mac today as far as I can verify.

Why does the card say 500GB when the repo is 393.5GB?

The card asks for 500GB of aggregated VRAM across a GPU cluster for the NVFP4 checkpoint. The roughly 106GB above the repo size is room for cache and runtime, which the card does not itemise.

Is the main checkpoint 4-bit or 8-bit?

Mostly 4-bit. It is the NVFP4 checkpoint, with NVFP4 experts in layers 0 to 39, MXFP8 experts in layers 40 to 47, and bf16 for attention, embeddings and vision. The Hub's "8-bit" tag comes from the FP8 tensors.

Does a 256GB Mac work with the MXFP8 repo or a smaller build?

No for the MXFP8 repo, which is 622.8GB. No smaller official build exists. The 256GB Mac Studio is 137.5GB short of the main repo before any cache.

Is the license Apache 2.0?

Yes. Both Hub repos declare Apache 2.0, and the announcement says the weights are released under it.

Sources

• Thinking Machines announcement of Inkling 2 Small, with the 12 November weights update
• Inkling 2 Small model card
• Hugging Face model card, NVFP4 repo
• Hub API, NVFP4 repo file sizes
• Hub API, MXFP8 repo file sizes
• hf_quant_config.json and config.json
• Apple Mac Studio tech specs
• llama.cpp pull request 25731