Last run 2026-11-15

Local LLM Speed Test

Name: ModelFit measured local LLM speed
Creator: ModelFit
License: https://creativecommons.org/licenses/by/4.0/

Real tokens/sec from readers' Macs and GPUs, grouped by model. A machine is only ranked against others on the same model, with the same prompts.

Machines: 11
Runs: 17
Fastest on Qwen3.8 27B: 150.0 tok/sNVIDIA GeForce RTX 5090

Test your machine

Run the speed test
$ npx @modelfit/cli@latest bench
No Node? brew tap modelfit-io/tap, brew trust modelfit-io/tap, brew install modelfit, then run modelfit bench. The first run downloads the test model it picks (about 3 to 17 GB).
Get your card
Your tok/s, your machine and your rank among the machines on the same model.
Post it on X and tag @modelfit_
Your result page has a Post on X button. Give your X handle and the card shows it.

Speed test card: NVIDIA GeForce RTX 5090, 150.0 tok/s on Qwen3.8 27B (Q4_K_M · 17.3 GB), spec. decoding · 4k ctx — A real card: NVIDIA GeForce RTX 5090, Nov 10.

It proposes the biggest test model your machine runs well, shows your result and asks before sending anything: machine type, memory, Ollama version and timings, plus your X handle if you give one (your card then shows it with your X profile picture). No name, no email, no files.

Method: Ollama, a warm-up, then 3 prompts of about 500 tokens, 256 generated tokens each, 4k context, temperature 0. A row is the median of that machine's runs. Needs Ollama.

Table 1Leaderboard

Measured speed by model

median tok/s

#	Machine	Memory	Decode	Prefill	Runs
Qwen3.8 27B qwen3.8:27b · Q4_K_M · 1 machine
1	NVIDIA GeForce RTX 5090spec. decoding · 4k ctx	32 GB VRAM	150.0 tok/s	1924.0 tok/s	1
Qwen3.5 9B Instruct qwen3.5:9b · Q4_K_M · 4 machines
1	NVIDIA GeForce RTX 4080	16 GB VRAM	92.5 tok/s	4902.8 tok/s	2
2	NVIDIA GeForce RTX 4070	12 GB VRAM	67.7 tok/s	3412.2 tok/s	1
3	NVIDIA GeForce RTX 5060 Ti	16 GB VRAM	67.1 tok/s	2721.6 tok/s	1
4	NVIDIA GeForce RTX 4060 Ti	16 GB VRAM	34.0 tok/s	1939.2 tok/s	2
Qwen3.5 4B Instruct qwen3.5:4b · Q4_K_M · 5 machines
1	NVIDIA GeForce RTX 5080	16 GB VRAM	163.8 tok/s	8304.6 tok/s	1
2	NVIDIA GeForce RTX 5060 Ti	16 GB VRAM	104.2 tok/s	4144.0 tok/s	1
3	NVIDIA GeForce RTX 4070	12 GB VRAM	101.4 tok/s	3543.3 tok/s	1
4	NVIDIA GeForce RTX 3070	8 GB VRAM	96.8 tok/s	2988.3 tok/s	2
5	NVIDIA GeForce RTX 5060	8 GB VRAM	69.8 tok/s	2476.9 tok/s	1
Ornith 1.0 9B ornith:9b · Q4_K_M · 1 machine
1	NVIDIA GeForce RTX 4070	12 GB VRAM	77.5 tok/s	3318.7 tok/s	1
Qwen2.5 1.5B Instruct qwen2.5:1.5b-instruct-q4_K_M · Q4_K_M · 3 machines
1	NVIDIA GeForce RTX 3090	24 GB VRAM	306.4 tok/s	-	1
2	NVIDIA GeForce RTX 4070 Ti	12 GB VRAM	260.1 tok/s	-	1
3	NVIDIA GeForce RTX 3060	12 GB VRAM	134.6 tok/s	-	1

Rows are ranked within their model only; the bar compares a machine with the fastest one on the same model. Decode is generation speed, prefill is prompt processing, both as Ollama reports them. Offload: part of the model ran outside GPU memory, which slows decode. MLX: Ollama ran its MLX build of the model instead of the GGUF file. Spec. decoding: the run decoded speculatively (several tokens per pass over the weights, at the CLI's 4k context), so it is ranked apart from classic runs of the same machine, which get their own row. A run faster than the machine's memory bandwidth allows for its model stays off the table. The Qwen2.5 1.5B Instruct group also keeps single-prompt runs from CLI 1.5.2 to 1.6, which report no prefill.

Fig. 1Cards

Latest cards

one per machine

Estimates vs measurements

How is this different from the speeds elsewhere on ModelFit?

The configurator, device pages and guides show estimated tokens/sec computed from memory bandwidth and model size. This table holds measurements: the same model, prompts and settings on every machine of a group. Where your machine and model have a row, the configurator shows the measured speed next to the estimate. The data is open: JSON export (CC BY 4.0). For quality scores, see the benchmark.