The quiet workhorse: 272 GB/s on a 128-bit bus — 3060-8G territory with Ada's efficiency, and the same 8 GB wall.
On paper the 4060 is behind its predecessor's memory subsystem — the 3060 8G runs 128-bit at 360 GB/s, this one 272 — so raw token speed lands within noise of a 3060 8G, not ahead of it. What Ada adds is clocks and efficiency: prompt processing is faster (~350–450 t/s on 8B), idle power drops to 115 W, and real-world streaming sits at ~38–50 t/s on 8B. For LLMs this card is the 3060 8G's successor in everything except the number that matters — the bus.
8 GB, as always: 8B at full quality (Q8 at 16K), 13B Q4 at short context (~20–28 t/s), 13B Q3 comfortable with the quality tax. MoE models with a few billion active are the other natural resident.
115 W is the lowest TDP on this site above the 4050. For a box that generates tokens overnight — batch work, embeddings, a personal assistant on standby — the wattage difference compounds into money. That is the 4060's actual LLM value: not speed, but cost-per-year.
The MTP question: if your model ships a built-in multi-token-prediction head
(Qwen 3.5/3.6/3.8, Gemma 4), the one-flag speculative pass
(--spec-type draft-mtp --spec-draft-n-max 2 --parallel 1) is worth
+40–70% (estimated) on token generation here; no 4060 A/B in the community record yet; the estimate follows the low-bandwidth end, where n-max 2 gains run largest. The price is ~0.6–2 GB of VRAM, the
output is bit-identical to the unaccelerated run, and models without MTP heads (Llama class) see
nothing. Full tuning rules in the MTP guide.
| Model | Size on GPU | Token gen | Token gen (MTP est.) | Prompt proc. | Fits? |
|---|---|---|---|---|---|
| 8B (Llama 3.1 8B, Mistral) | ~4.9 GB | ~38–50 t/s | ~53–85 t/s | ~350–450 t/s | ✅ Q8_0, 16K+ ctx |
| 13B Q4_K_M | ~8.5 GB | ~20–28 t/s | ~28–48 t/s | ~300–400 t/s | ⚠️ tight, short ctx only |
| 13B Q3_K_M | ~6.7 GB | ~23–31 t/s | ~32–53 t/s | ~350 t/s | ✅ with quality loss |
| 27B | ~19 GB | — | — | — | ❌ no |
MTP est. = rough prior with --spec-type draft-mtp for models that ship MTP heads (Qwen 3.5/3.6/3.8, Gemma 4); Llama-class models get no MTP speedup. Actual gains depend on model, quant, context length, llama.cpp build and your card — sweep --spec-draft-n-max, don't trust the column blindly. MTP guide →
Figures assume full offload (-ngl 99), ~8K context and batch 2048. Token speed tracks the memory bus almost linearly, so treat ranges as class estimates — your llama.cpp build and the model's attention layout shift them.
| Quant | GGUF size | 1× 4060 (8 GB) | 2× 4060 (16 GB) |
|---|---|---|---|
| Q4_K_M | ~19 GB | ❌ ~11 GB over | ❌ ~3 GB over |
| Q5_K_M | ~22 GB | ❌ | ❌ |
| Q6_K | ~25 GB | ❌ | ❌ |
| Q8_0 | ~33 GB | ❌ | ❌ |
The 27B tier is unreachable at any quant until 4× (32 GB), where Q4_K_M manages roughly ~8–11 t/s at 4–8K context. Everything below that is offload territory — a few tokens per second, which is what offload is.
-c 4096 works; don't promise anyone 13B at 32K.PCIe 4.0 only — Ada never shipped NVLink — so multi-card pays the ~25–35% interconnect tax, with 8 GB tiles on top.
| Build | GPU power | VRAM | 13B Q4 | 27B Q4_K_M |
|---|---|---|---|---|
| 2× 4060 | 230 W | 16 GB | ~27–36 t/s | ❌ ~3 GB over |
| 4× 4060 | 460 W | 32 GB | ~45–60 t/s | ~8–11 t/s, 4–8K ctx |
Four 4060s (460 W, 32 GB) is a legitimate budget 27B-Q4 box — slower than the spec sheet suggests only because the split tax is real. Slot count is the constraint: a board with four working lanes is workstation territory.
When a model won't fit, -ngl N parks the tail layers on the CPU. The mental model
that saves pain: offloaded layers don't run at a fraction of GPU speed — they run at CPU
speed. Handy moves: peel the last 2–4 layers off to reclaim ~0.5–1 GB of VRAM for almost
nothing; keep MoE experts on CPU with --n-cpu-moe; park the output embedding with
-ot output=CPU on big models. Budget 32 GB system RAM for 27B-class offload, 64 GB for
70B. And never offload the KV cache — the session becomes unusable.
llama-server -m 27b-q4_k_m.gguf -ngl 99 -sm layer -c 8192 --tensor-split 1,1,1,1
llama-server -m llama-3.1-8b-q6_k.gguf -ngl 99 -c 8192 -b 2048 -ub 2048 --host 127.0.0.1 --port 8080
The 4060 is the least exciting 8 GB card on the site and a very good low-power one: 8B at full quality, 115 W, silent. Buy it for the electricity bill and the small-model lane. Buy anything 16 GB the moment 13B stops being a compromise.