Can you run Qwen3.5 35B-A3B Instruct on RTX 4060?
Qwen3.5 35B-A3B Instruct Q4_K_M on the NVIDIA GeForce RTX 4060: verdict, VRAM math and estimated speed.
- VRAM
- 8 GB(7.2 usable)
- Model weights
- 20 GBQ4_K_M
- Decode speed
- est. 30 tok/srough estimate, offload not measured
- First token
- ~0.2s
No, Qwen3.5 35B-A3B Instruct does not realistically run on the NVIDIA GeForce RTX 4060. 20 GB weights at Q4_K_M vs 7.2 GB usable VRAM; est. 30 tok/s, rough estimate, offload not measured (ModelFit, 2026).
→ Smallest tracked card that runs it: AMD Radeon RX 7900 XTX (24 GB)
- Verdict
- Does not fit
- Decode speed
- est. 30 tok/s, rough estimate, offload not measured
- Weights
- 20 GB Q4_K_M
VRAM math and speed are ModelFit engine estimates, not measurements. Typical error: the median gap between the ModelFit speed model and measured runs on hardware the model was not fitted on, for this backend and model size. Commands are registry-verified Ollama tags.
Cite this page: ModelFit, Qwen3.5 35B-A3B Instruct on RTX 4060, https://modelfit.io/can-i-run/qwen3.5-35b-a3b-q4-on-rtx-4060/, updated November 2026, CC BY 4.0.
Get told when a better model fits your RTX 4060
Get one email when a new open-weight model is a better fit for this machine.
You'll also get the weekly ModelFit email. By signing up you agree to our Privacy Policy. Unsubscribe from either email anytime, in one click.
What limits this combo
The four constraints the fit engine checks for Qwen3.5 35B-A3B Instruct on the RTX 4060, in order of what usually breaks first.
- Weights fit
- Blocked20 GB of weights against 7.2 GB usable: the model does not load fully in VRAM; the shortfall spills to system RAM.
- Context ceiling
- BlockedThe weights alone exceed the budget, so no usable context fits on top.
- Speed
- Tightest. 30 tok/s, rough estimate, offload not measured: fine for chat, below the ~30 tok/s that agentic coding loops need to feel responsive.
- Use-case ceiling
- BlockedNothing is graded above D: the model does not realistically fit this hardware.
Where the RTX 4060 sits for Qwen3.5 35B-A3B Instruct
The RTX 4060 cannot hold Qwen3.5 35B-A3B Instruct in VRAM: 20 GB of weights against 7.2 GB usable. The smallest tracked card that runs Qwen3.5 35B-A3B Instruct fully in VRAM is the RTX 3090 (24 GB, est. 212 tok/s, typical error 15%). Context ceiling on this card: none; on the RTX 3090: 64k.
| Card | VRAM | Usable | Verdict | Decode speed | Max context |
|---|---|---|---|---|---|
| RTX 4060 (this card) | 8 GB | 7.2 GB | No | est. 30 tok/srough estimate, offload not measured | none |
| RTX 3060 | 12 GB | 10.8 GB | No | est. 35 tok/srough estimate, offload not measured | none |
| RTX 4070 | 12 GB | 10.8 GB | No | est. 51 tok/srough estimate, offload not measured | none |
| RTX 4060 Ti | 16 GB | 14.4 GB | Tight | est. 67 tok/srough estimate, offload not measured | none |
| RTX 4070 Ti SUPER | 16 GB | 14.4 GB | Tight | est. 136 tok/srough estimate, offload not measured | none |
| RTX 5070 Ti | 16 GB | 14.4 GB | Tight | est. 173 tok/srough estimate, offload not measured | none |
| RTX 5080 | 16 GB | 14.4 GB | Tight | est. 181 tok/srough estimate, offload not measured | none |
| RTX 3090 | 24 GB | 21.6 GB | Yes | est. 212 tok/s | 64k |
| RTX 4090 | 24 GB | 21.6 GB | Yes | est. 242 tok/s | 64k |
| RTX 5090 | 32 GB | 28.8 GB | Yes | est. 350 tok/s | 128k |
Same engine as the verdict above: 90% of VRAM usable, KV-cache at fp16, decode speed from the ModelFit speed model, typical error 15%. What typical error means. Verdicts of the other cards are computed for Qwen3.5 35B-A3B Instruct Q4_K_M exactly.
Quant explorer
Switch between the quality-gated builds tracked for this exact combo. Q4_K_M stays the default recommendation; heavier Q6/Q8 builds only appear when their exact Ollama tags are registry-verified.
| Quant | Load | Verdict | Fit | Speed | First token | Safe context | Run |
|---|---|---|---|---|---|---|---|
| 20 GB | No | No fit | est. 30 tok/srough estimate, offload not measured | ~0.2s · Instant | none | ||
| 38.7 GB | No | No fit | est. 18 tok/srough estimate, offload not measured | ~1.2s · Instant | none |
Why no 2-bit builds: Q1/Q2-class quants can look attractive in a memory table, but their quality loss is large enough that ModelFit excludes them from rankings instead of inflating the catalog with junk options. Usable VRAM: 7.2 GB.
Workload verdicts
ModelFit engine estimateQwen3.5 35B-A3B Instruct on the RTX 4060, graded per use case from the model's registry-verified tuning and the engine's speed estimate for this exact combo (est. 30 tok/s, rough estimate, offload not measured).
- Chat
- DQwen3.5 35B-A3B Instruct does not realistically fit this hardware.
- Coding
- DQwen3.5 35B-A3B Instruct does not realistically fit this hardware.
- Agentic coding
- DQwen3.5 35B-A3B Instruct does not realistically fit this hardware.
- Reasoning
- DQwen3.5 35B-A3B Instruct does not realistically fit this hardware.
- RAG
- DQwen3.5 35B-A3B Instruct does not realistically fit this hardware.
Grades combine tag-verified tuning (a model not built for the workload caps at C) with the tok/s floor each workload needs to feel usable. A combo that partially offloads caps at C; one that does not fit is D everywhere.
Memory math: weights + context vs budget
Weights take 20 GB. Context costs extra KV-cache on top; this is where long-context sessions break on cards that technically fit the weights.
This configuration exceeds the comfortable VRAM budget by about 13 GB. Expect offload pressure or a shorter safe context.
| Context | KV-cache | Total | Fits |
|---|---|---|---|
| 8k tokens | 0.2 GB | 20.2 GB | Over |
| 16k tokens | 0.3 GB | 20.3 GB | Over |
| 32k tokens | 0.6 GB | 20.6 GB | Over |
| 64k tokens | 1.3 GB | 21.3 GB | Over |
| 128k tokens | 2.5 GB | 22.5 GB | Over |
KV-cache figures assume an fp16 cache, the llama.cpp/Ollama default. Standard GQA models use a size-class estimate (8 KV heads x 128 head dim class); hybrid linear-attention models (Qwen3.5/3.6, Qwen3-Next) use the exact per-token cost from their published config, since only their sparse full-attention layers cache KV. A q8_0 KV cache roughly halves either figure. Estimates, not measurements.
See how fast it feels
Qwen3.5 35B-A3B Instruct: ~0.2s to start, then est. 30 tok/s, rough estimate, offload not measured.
# Press Send to run the fixed message below.
# fixed message, the same for every model
>>> Summarize these meeting notes in one short paragraph: what was decided, and who does what next.
[meeting-notes.txt]
Weekly sync, support tools team Attendees: Maya (lead), Jonas, Priya, Tom, Elena Agenda: local drafting assistant pilot, knowledge base cleanup, customer feedback survey, on-call rota, next steps 1. Local drafting assistant pilot - Maya recapped the goal: agents draft replies with an assistant that runs on the team's own Macs, so customer emails never leave the building and nothing depends on an outside service being up. - Jonas ran the pilot for two weeks on four machines. Agents used it for first drafts on roughly a third of tickets. Most drafts needed light edits. A few needed a full rewrite, mostly billing questions where the model guessed at policy instead of saying it did not know. - Priya said the biggest complaint was the wait before the first word on long tickets. Short tickets felt instant. Tickets with a long email thread pasted in took several seconds before anything appeared, and agents read that as the tool being stuck. Two agents restarted the app because of it. - Tom suggested trimming the thread to the last three messages before sending it to the model. Elena worried that would drop context the customer refers back to, like an order number from the first email. Agreed to test both on the same set of tickets before choosing. - Jonas said the speed once the reply starts was fine on every machine. Agents read slower than the model writes, so nobody asked for faster output. The only request was a visible sign that the model is reading, so people stop restarting it. - Memory: the oldest machine, the one with the least memory, could not keep the model loaded next to the browser and the ticketing app. It swapped and slowed down badly by mid-morning. The other three machines had no issue all day. - Priya asked whether a larger model would fix the billing guesses. Jonas thought not: the model guesses because the policy text it needs is missing or contradictory, and a bigger model would guess more fluently, not less. - Decision: keep the current model size for now. Revisit only after the knowledge base cleanup, and only if the edit rate on billing drafts is still high. 2. Knowledge base cleanup - Elena found about forty articles that contradict each other on refund windows and plan changes. These are the likely source of the billing guesses. - Priya proposed one owner per section, with a review date on every article so stale pages show up on their own instead of during an angry ticket. - Agreed: Elena owns billing and refunds, Priya owns accounts and login, Tom owns shipping and returns. Each owner archives duplicates, merges the rest, and flags anything that needs a policy decision from Maya. - Jonas will add the cleaned articles to the assistant's reference folder once each section is signed off, not before. Mixing old and new versions would make the guessing worse. - Tom asked for a template for new articles. Elena will share the one billing already uses. 3. Customer feedback survey - Priya shared the quarterly survey results. Response time scores went up since the last quarter, and the free-text comments mention faster first replies more than once. - Several comments still complain about being asked for the same order details twice. Elena thinks this happens when a ticket moves between agents and the notes field is left empty. - Agreed: the handover checklist gets one new line, fill in the notes field before reassigning a ticket. Tom will update the checklist and post it in the team channel. - Maya wants the next survey to ask one question about reply quality, not only speed, so the team can see whether the assistant drafts change how customers rate answers. 4. On-call rota - Two people asked to swap weeks in November. Maya approved both swaps. - Tom raised that weekend coverage gets thin when two people are on leave at the same time. Agreed that leave requests go in the team channel at least two weeks ahead, and Maya checks the rota before approving. - No change to escalation rules. The escalation doc is still correct. 5. Next steps - Jonas: run the trimmed thread versus full thread test on fifty past tickets and report the wait before the first word and the edit rate for each. - Jonas: add a reading indicator to the assistant panel so agents can see the model is working on a long ticket. - Elena, Priya, Tom: finish their knowledge base sections and send Maya the open policy questions. - Maya: ask IT whether the oldest machine can get more memory or should be replaced at the next refresh. - Everyone: log any draft that invents a policy, with the ticket link, in the shared sheet. - Next sync: same time next week. Maya sends the agenda the day before.
Estimated speed, not a benchmark. Model loading excluded.
Upgrade path
The smallest tracked card that runs Qwen3.5 35B-A3B Instruct comfortably is the AMD Radeon RX 7900 XTX (24 GB VRAM).
Cards that run Qwen3.5 35B-A3B Instruct comfortably
engine estimate- Smallest that runs itRTX 3090 · 24 GB VRAMRuns Qwen3.5 35B-A3B Instruct (Q4_K_M) at est. 212 tok/s in 24 GB of VRAM.View on Amazon
- RTX 4090 · 24 GB VRAMRuns Qwen3.5 35B-A3B Instruct (Q4_K_M) at est. 242 tok/s in 24 GB of VRAM.View on Amazon
- RTX 5090 · 32 GB VRAMRuns Qwen3.5 35B-A3B Instruct (Q4_K_M) at est. 350 tok/s in 32 GB of VRAM.View on Amazon
ModelFit may earn a commission on purchases through these links, at no extra cost to you.
Qwen3.5 35B-A3B Instruct on RTX 4060: FAQ
Can the NVIDIA GeForce RTX 4060 run Qwen3.5 35B-A3B Instruct?
Not realistically. Qwen3.5 35B-A3B Instruct (Q4_K_M) needs about 20 GB against 7.2 GB usable VRAM on the RTX 4060; even with heavy CPU offload it would run at unusable speed.
How much VRAM does Qwen3.5 35B-A3B Instruct need?
About 20 GB for the weights at Q4_K_M, plus KV-cache for context: roughly 0.3 GB extra at 16k tokens. The RTX 4060 budget is 7.2 GB (8 GB x 90%).
What is the best quantization of Qwen3.5 35B-A3B Instruct for the RTX 4060?
Stick with the Q4_K_M build at 20 GB; every heavier quant exceeds the 7.2 GB usable VRAM.
What GPU do I need to run Qwen3.5 35B-A3B Instruct comfortably?
The smallest tracked card that runs Qwen3.5 35B-A3B Instruct (Q4_K_M) fully in VRAM is the AMD Radeon RX 7900 XTX (24 GB).
How much context can Qwen3.5 35B-A3B Instruct use on the RTX 4060?
None worth having: the 20 GB of weights alone exceed the 7.2 GB usable VRAM, so the model only runs with system-RAM offload and short prompts.
Is Qwen3.5 35B-A3B Instruct on the RTX 4060 fast enough for coding agents?
No: the model does not realistically fit this card, so agentic loops are out of the question on it.
Qwen3.5 35B-A3B Instruct on other hardware
Other models on the RTX 4060
- Qwen3 8B Q4_K_M
- Qwen3 14B Q4_K_M
- GPT-OSS 20B MXFP4
- Qwen3.8 27B Q4_K_M
- Llama 3.3 70B Instruct Q4_K_M
- Gemma 4 26B-A4B Q4_K_M
- Qwen3.5 4B Instruct Q4_K_M
- Gemma 4 E4B Q4_K_M
- Gemma 3 4B Instruct Q4_K_M
- Llama 3.1 8B Instruct Q4_K_M
- Qwen3.5 9B Instruct Q4_K_M
- Gemma 4 12B Q4_K_M
- Mistral Nemo 12B Q4_K_M
- Qwen3.5 9B Instruct (Q8) Q8_0
- Gemma 4 12B (Q8) Q8_0
- LFM2 24B-A2B Instruct Q4_K_M
- Qwen3.5 27B Instruct Q4_K_M
- Qwen3.6 27B Q4_K_M
- Gemma 4 31B Q4_K_M
- Qwen3.6 35B-A3B Q4_K_M
- Qwen3.8 27B (Q8) Q8_0