NVIDIA Dory 1.0 4B on a Mac: 3.7GB Fits 16GB, Fork Build Needed

NVIDIA published NVIDIA-Nemotron-Labs-Dory1.0-4B on October 16, 2026, and the memory question is settled before it starts: the smallest official build is a 3.673 GB quantized GGUF. That file fits inside the share of unified memory macOS hands to the GPU, even on an 8GB Mac. The question that actually decides whether you can run it is software. No llama.cpp release reads the Dory architecture, Ollama has no tag for it, and NVIDIA's own documented path for Apple Silicon is a build from a pinned revision of a contributor fork. That is a different kind of blocker from the one in our Nemotron 3.5 Bolt breakdown, where a 74B model needed a 36GB Mac and Metal support was missing too.

What is NVIDIA Dory 1.0 4B?

Dory1.0-4B is a small looped language model and the first Dory release NVIDIA has published. The GGUF model card describes it as a looped (latent-recurrent) normalized-GPT with interleaved sliding-window and full attention: 4B parameters, up to 256K tokens of context, English only, under the OpenMDW 1.1 license, released October 16, 2026.

Looped inference means the layer stack is split into three groups: an Input block of 8 layers, a shared Recurrent block of 20 layers that is applied several times, and an Output block of 8 layers, with the input reinjected into every loop. The loop count is the number of times that shared block runs, and it is the interesting part of this model. Changing dory.num_loops between 1, 2, 3 and 4 trades compute per token for accuracy without downloading a second checkpoint. The card's wording is that one checkpoint serves several inference compute budgets, with no retraining and no model swap.

The shipped Q5_K_M file carries the shape in its own header: 72 blocks, an embedding length of 2,560, 16 attention heads with 4 key-value heads of 256 elements each, a sliding window of 1,024 tokens, a maximum context of 262,144 tokens, and 36 attention layers of which 9 are full attention. ModelFit read those values directly from the published file header to fill the memory table below.

How much memory does Dory 1.0 4B need on a Mac?

Less than any current Mac offers. Sizes below are byte counts read from the Hugging Face blobs API on October 19, 2026, converted to decimal gigabytes (10^9 bytes) and megabytes (10^6 bytes):

The number that matters is inside the first row, not the row itself. The repository total of 3,675 MB counts the card, five images and the LFS pointer files too; the file you download is NVIDIA-Nemotron-Labs-Dory1.0-4B-Q5_K_M.gguf at exactly 3,673,288,864 bytes (3.673 GB, or 3,673 MB). The card publishes its SHA-256, 96080be49ffd67b916018b7bdef8c484762576fb52780e10ab98ac4978a660e0, and the file NVIDIA's own Mac instructions expect is that one file and nothing else. Within the BF16 repository the single model.safetensors blob is 7,950,515,472 bytes, and the MXFP8 and NVFP4 repositories ship one safetensors blob each.

Multiply that by the engine rule and the table stops being a table. ModelFit does not give a model all of a Mac's RAM: the budget is 70% of unified memory at 32GB and below, ramping linearly to 85% at 128GB and above, because macOS caps how much memory the GPU can wire. That is the same rule behind every pick in the homepage wizard and the same math as in the VRAM cheat sheet.

The 8GB row deserves the asterisk it gets. Weights are not the whole footprint: weights plus the KV cache plus the runtime's compute buffers have to sit inside that 5,600 MB budget, and the leftover after the weights is 1,927 MB. That is enough for a 4,096-token context at the settings below, not for 256K. If you are on an 8GB Mac, the 8GB Mac guide covers the trade in detail. On 16GB, Dory is not the model that pushes you out of your tier, which is why it does not change the picks in our 16GB Mac roundup.

Does Dory 1.0 4B run in llama.cpp or Ollama today?

No to both, and the two answers come from the same place: the architecture is new and nothing shipped reads it yet.

• llama.cpp: pull request 30306, titled "dory support for llama.cpp", was still open and in draft on October 19, 2026. It was opened on October 10, last updated on October 17, and stands at 4 commits touching 17 files. Until it merges, no llama.cpp release, including the 0.4.x line, loads a Dory GGUF.
• Ollama: ollama.com/library/dory returns 404. Neither nemotron-labs-dory nor nemotron-dory exists either, so there is no ollama run command to copy. The 2026 runtime landscape in MLX vs Ollama assumes a tag exists; here it does not.
• vLLM: the NVIDIA-heritage path for the NVFP4 and MXFP8 builds, pull request 60149 "Add Dory support", is also open and unmerged, so the card tells you to install vLLM from that pull request.
• MLX: mlx-lm has no Dory model implementation, and the mlx-community organisation has published no Dory conversion. MLX is not a way around the fork build today.

What remains is the path NVIDIA documents for Apple Silicon: clone a contributor fork, check out one exact revision, build with Metal, and run the server yourself.

How do you build the Dory fork on Apple Silicon?

The GGUF model card is explicit that support is not in a release and that you should build from the Dory-enabled source rather than follow the moving pull request head. It pins revision 98a74c97ea09ac72f6ceda1559808fc0a2f447ce of brianchmielNV/llama.cpp, which exists and dates from October 11, 2026. The build is native ARM64 macOS, not Intel and not Rosetta, and it expects Homebrew plus Apple's Command Line Tools:

brew install cmake openssl@3

git init llama.cpp-dory
cd llama.cpp-dory
git remote add origin https://github.com/brianchmielNV/llama.cpp.git
git fetch --depth 1 origin 98a74c97ea09ac72f6ceda1559808fc0a2f447ce
git checkout --detach FETCH_HEAD

HF_UI_VERSION=b11551 cmake -S . -B build \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_OSX_ARCHITECTURES=arm64 \
    -DCMAKE_OSX_SYSROOT="/Library/Developer/CommandLineTools/SDKs/MacOSX26.sdk" \
    -DGGML_METAL=ON -DGGML_METAL_EMBED_LIBRARY=ON \
    -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=Apple \
    -DLLAMA_BUILD_SERVER=ON \
    -DLLAMA_BUILD_TESTS=OFF -DLLAMA_BUILD_EXAMPLES=OFF -DLLAMA_BUILD_APP=OFF \
    -DLLAMA_BUILD_UI=OFF -DLLAMA_USE_PREBUILT_UI=ON \
    -DLLAMA_OPENSSL=ON -DOPENSSL_ROOT_DIR="$(brew --prefix openssl@3)"
cmake --build build --parallel 4 --target llama-server

Three practical notes before you spend twenty minutes on a compile. The -DCMAKE_OSX_SYSROOT path has to match the SDK installed on your machine, and it changes with every Xcode release, so check /Library/Developer/CommandLineTools/SDKs/ first. The build pulls a prebuilt browser UI, so no Python and no JavaScript toolchain is needed. And verification comes before the launch: the card gives the exact expected size and a checksum command, which is the only way to know the quantization you downloaded is the one NVIDIA published.

echo "96080be49ffd67b916018b7bdef8c484762576fb52780e10ab98ac4978a660e0  dory-q5_k_m.gguf" | shasum -a 256 -c -

The launch command from the same card starts a local server with two loops, reasoning on, an f16 KV cache and a 4,096-token context:

./build/bin/llama-server \
    -m ./dory-q5_k_m.gguf \
    --alias dory \
    --host 127.0.0.1 --port 8080 \
    -ngl 999 \
    --ctx-size 4096 --parallel 1 \
    --batch-size 512 --ubatch-size 512 \
    --cache-type-k f16 --cache-type-v f16 \
    --flash-attn off \
    --no-context-shift \
    --jinja --reasoning on \
    --temp 1.0 --top-p 0.95 --top-k 0 --min-p 0 --repeat-penalty 1 \
    --override-kv "dory.num_loops=int:2,dory.recurrent_kv_cache_mode=str:per_loop"

The server is local-only and unauthenticated, so do not expose port 8080. Its OpenAI-compatible endpoint is http://127.0.0.1:8080/v1 under the alias dory, which means your existing client configuration works unchanged. Note what the recipe asks for: 4,096 tokens of context, not 256K, and no offload tricks. This is a plain llama.cpp build, so nothing in the 0.4.0 lazy tensor and sparse attention changes applies until the pull request merges upstream.

One footnote the card keeps separate from the numbers: the benchmark table is measured on BF16, while the GGUF is a quantized export. The card states that outright in the download section. Treat the scores below as NVIDIA's BF16 results for the architecture, not as measurements of the Q5_K_M file.

What does the loop count cost in memory?

This is the part of Dory that a Mac owner has to budget for, because two settings interact: the loop count and the KV cache mode. NVIDIA's card documents both. Per-loop caching is the default in the reference command, and it stores a separate KV cache for every loop, so KV cache memory grows with the loop count. The alternative, last_loop, keeps a single cache per recurrent attention layer where each loop overwrites the previous one. The card describes it as carrying no KV-cache overhead relative to an equivalent non-looped model, for any loop count.

ModelFit's arithmetic from the published header, at the f16 cache the reference command asks for:

Each attention layer costs 4,096 bytes per token at 16-bit (4 key-value heads times 256 elements times 2 bytes, for keys and values). Dory has 36 attention layers of its 72 blocks: 8 in the Input block, 20 in the shared Recurrent block and 8 in the Output block. Under last_loop the recurrent block is stored once. Under per_loop every loop gets its own cache, so the layer count becomes 16 plus 20 times the loop count, which is 56 at two loops and 96 at four.

The header also carries dory.num_loops = 4 and dory.recurrent_kv_cache_mode = per_loop, so the loop count is a runtime override rather than a property of your download. NVIDIA's card recommends two loops as the starting point and tells you to set the value in --override-kv.

Those per-token figures are an upper bound for the sliding portion of the model. 27 of the 36 attention layers use a sliding window of 1,024 tokens, and a runtime is free to keep less than a full-context cache for them. They also exclude compute buffers and everything else the runtime allocates. The full-attention layers dominate at length, and last_loop does not shrink them. Nine full-attention layers times 262,144 tokens times 4,096 bytes per token is 9,664 MB of KV cache at f16. That is why NVIDIA's own Mac recipe stops at 4,096 tokens, and why the 256K figure on the card is a capability rather than a plan. Size your own mix of context length and loop count in the LLM memory calculator, or read the tier math in the RAM guide for local LLMs.

How does the loop count change NVIDIA's published scores?

Looping is the point of the model, and the card's table shows the gain is front-loaded. Each Dory column below is the same checkpoint at a different loop count, from the GGUF model card, measured by NVIDIA with NeMo Gym and the Nemo Evaluator SDK:

The second loop buys most of the agentic gain: SWE-bench Verified moves 9.2 points from one loop to two, then 2.3 points more at four loops, and Terminal-Bench 2.1 behaves the same way. Reasoning and instruction following gain less, and AA-Omniscience accuracy is flat across all three settings. NVIDIA's guidance follows its own table: two loops as the default, one loop when compute per token is the priority, four loops when accuracy matters more, especially for agentic work.

The card also argues that looping is token-efficient, meaning the model solves the same task with fewer generated tokens as the loop count rises, with the compute going into latent thinking instead of a longer answer. That is a claim about the model, not about your machine, and it carries an obvious warning. A higher loop count spends more compute per token, so end-to-end latency can move against you even as the output gets shorter. NVIDIA's advice is to measure end-to-end latency on your own workload, and ModelFit has no Mac measurement of Dory to offer in its place. ModelFit runs no benchmarks, and no third-party Apple Silicon numbers for this model were published as of October 19, 2026, which is why no tokens per second figure appears in this article.

What if you have an NVIDIA GPU instead?

The other three repositories are the paths NVIDIA actually documents for its own hardware. The NVFP4 build is the flagship: the card says Dory was pre-trained natively in NVFP4, NVIDIA's 4-bit floating-point format, before continuing in BF16. The NVFP4 card shows a vLLM serve command with --max-model-len 262144, --max-num-seqs 128 and the same two-loop recommendation. MXFP8 and BF16 are the wider-compatibility options at 4.770 GB and 7.969 GB. All three still route through an unmerged vLLM pull request as of October 19, so the same "not in a release yet" caveat applies on Blackwell, just with fewer steps.

FAQ

Can a 16GB Mac run NVIDIA Dory 1.0 4B?

Yes, on memory. The Q5_K_M GGUF is 3,673 MB and the engine budget on a 16GB Mac is 11,200 MB, leaving 7,527 MB for the KV cache and compute buffers. An 8GB Mac also fits the weights with 1,927 MB to spare, which covers a short context but not the model's full window. The limit is not memory: no released runtime loads the architecture.

Does Ollama have a Dory tag?

No. ollama.com/library/dory returned 404 on October 19, 2026, and no variant tag such as nemotron-labs-dory or nemotron-dory exists. There is no ollama run command for this model.

What exactly do I download, and how do I verify it?

NVIDIA-Nemotron-Labs-Dory1.0-4B-Q5_K_M.gguf from the GGUF repository, which is 3,673,288,864 bytes. Verify it against the published SHA-256 96080be49ffd67b916018b7bdef8c484762576fb52780e10ab98ac4978a660e0 with shasum -a 256 -c - before you run anything. The repository total of 3,675 MB includes the card and images, so it is not the download size.

Does a 4-loop setting cost more memory than 2 loops?

It depends on the cache mode, not on the weights. Under per_loop, which is what NVIDIA's reference command uses, the cache per token rises from 229,376 bytes at two loops to 393,216 bytes at four. Switch the value in --override-kv to last_loop and the cache per token stays at 147,456 bytes whatever the loop count, at the cost of a cache that holds only the final loop's keys and values.

Is there an MLX or LM Studio path for Dory on Apple Silicon?

Not an official one. mlx-lm has no Dory implementation and the mlx-community organisation published no conversion, so MLX cannot load the architecture today. LM Studio and other GUI wrappers sit on llama.cpp or MLX, so they inherit the same gap. The only documented Apple Silicon path is the pinned fork build above, which uses Metal through llama.cpp.

Sources

• NVIDIA-Nemotron-Labs-Dory1.0-4B-GGUF model card and GGUF build instructions (NVIDIA, 2026)
• NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16, NVFP4 and MXFP8 repositories (NVIDIA, 2026)
• llama.cpp pull request 30306: dory support and the pinned contributor fork (GitHub, 2026)
• vLLM pull request 60149: Add Dory support (GitHub, 2026)
• Ollama library: dory (404 as of October 19, 2026)
• mlx-lm model implementations (MLX, 2026)