The budget LLM pick of Ada. 16 GB at a fraction of 3090 money — 13B at Q6, MoE 30B, a 27B Q3 demo — all on a slow 128-bit bus.
Double the 8 GB sibling's memory and the model list stops being a small-model list:
288 GB/s is the slowest bus on the 16 GB tier — a 5060 Ti 16G has the same width at 1.55× the speed, and a 4070 Ti Super has 2.3×. Token generation here is fine for the models that fit (21–29 t/s on 13B Q4); it is not fast. The value math is VRAM-per-dollar, and on that score this card is the series' best.
160 W for a 16 GB card. In a 24/7 box the 4060 Ti 16G sips where the 3090 gulps — a real difference when the card is the server.
The MTP question: if your model ships a built-in multi-token-prediction head
(Qwen 3.5/3.6/3.8, Gemma 4), the one-flag speculative pass
(--spec-type draft-mtp --spec-draft-n-max 2 --parallel 1) is worth
+40–70% (estimated) on token generation here; no per-card A/B in the record yet; slow-bus cards historically gain the most at n-max 2. The price is ~0.6–2 GB of VRAM, the
output is bit-identical to the unaccelerated run, and models without MTP heads (Llama class) see
nothing. Full tuning rules in the MTP guide.
| Model | Size on GPU | Token gen | Token gen (MTP est.) | Prompt proc. | Fits? |
|---|---|---|---|---|---|
| 8B Q8_0 | ~9.7 GB | ~40–54 t/s | ~56–92 t/s | ~400–550 t/s | ✅ 32K ctx |
| 13B Q4_K_M | ~8.5 GB | ~21–29 t/s | ~30–49 t/s | ~450–550 t/s | ✅ 32K+ ctx |
| 13B Q6_K | ~13.8 GB | ~16–22 t/s | ~23–37 t/s | ~400–500 t/s | ✅ 8K ctx |
| 27B Q3_K_M | ~15.8 GB | ~6–8 t/s | ~9–14 t/s | ~300 t/s | ⚠️ 4K ctx |
| MoE 30B-A3B (Q3) | ~14.5 GB | ~23–31 t/s | ~33–53 t/s | ~400–500 t/s | ⚠️ 4–8K ctx, 3B active |
MTP est. = rough prior with --spec-type draft-mtp for models that ship MTP heads (Qwen 3.5/3.6/3.8, Gemma 4); Llama-class models get no MTP speedup. Actual gains depend on model, quant, context length, llama.cpp build and your card — sweep --spec-draft-n-max, don't trust the column blindly. MTP guide →
Figures assume full offload (-ngl 99), ~8K context and batch 2048. Token speed tracks the memory bus almost linearly, so treat ranges as class estimates — your llama.cpp build and the model's attention layout shift them. MoE speed tracks active parameters, not total size — the 30B-A3B row is the card's headline trick.
| Quant | GGUF size | 1× 4060 Ti 16G (16 GB) | 2× 4060 Ti 16G (32 GB) |
|---|---|---|---|
| Q3_K_M | ~15.8 GB | ⚠️ ~6–8 t/s, 4K ctx | ✅ (overkill) |
| Q4_K_M | ~19 GB | ❌ ~3 GB over | ✅ ~11–14 t/s, 8K ctx |
| Q5_K_M | ~22 GB | ❌ | ✅ ~9–12 t/s, 4–8K ctx |
| Q6_K | ~25 GB | ❌ | ✅ ~8–10 t/s, 4–8K ctx |
| Q8_0 | ~33 GB | ❌ | ❌ ~1 GB over |
Single-card 27B means Q3 at 4K — usable in a pinch, clearly quant-limited. The 2-card build (320 W, 32 GB) is the series' value 27B rig: Q4 at a modest pace, Q5/Q6 at short context, and MoE 30B at full Q4 (~18 GB) streaming at ~30–38 t/s. Slower than a 3090 pair at the same models, cheaper to build and run.
-c 4096–8192 — the plain Q8_0 file is 1.4 GB too big.-c 4096 plus quantized KV (--cache-type-k q8_0 --cache-type-v q8_0) to make the fit.PCIe 4.0, ~25–35% split tax. 16 GB tiles mean a pair lands on the 27B Q4 line (32 GB).
| Build | GPU power | VRAM | 27B Q4_K_M | 27B Q5_K_M | MoE 30B Q4 |
|---|---|---|---|---|---|
| 2× 4060 Ti 16G | 320 W | 32 GB | ~11–14 t/s, 8K ctx | ~9–12 t/s, 4–8K ctx | ~30–38 t/s |
The pair is the card's reason to exist in plural: 32 GB at 320 W, no workstation board required — two x16 (or x8/x8) slots on a normal ATX close the job.
When a model won't fit, -ngl N parks the tail layers on the CPU. The mental model
that saves pain: offloaded layers don't run at a fraction of GPU speed — they run at CPU
speed. Handy moves: peel the last 2–4 layers off to reclaim ~0.5–1 GB of VRAM for almost
nothing; keep MoE experts on CPU with --n-cpu-moe; park the output embedding with
-ot output=CPU on big models. Budget 32 GB system RAM for 27B-class offload, 64 GB for
70B. And never offload the KV cache — the session becomes unusable.
llama-server -m 27b-q4_k_m.gguf -ngl 99 -sm layer -c 8192 --tensor-split 1,1
llama-server -m llama-3.1-13b-q4_k_m.gguf -ngl 99 -c 32768 -b 2048 -ub 2048 --cache-type-k q8_0 --cache-type-v q8_0 --host 127.0.0.1 --port 8080
The RTX 4060 Ti 16G is Ada's LLM card, full stop: the cheapest door into 16 GB, the 13B quality tier, MoE models and 27B-at-Q3, at 160 W. The 128-bit bus caps its speed where the 3090-class cards soar — but nobody at this price point is paying for speed, they're paying for the models that fit. And it fits the most of any card below it.