The default 13B card: 504 GB/s on a 192-bit bus with 12 GB — full-quality 13B with a real context, at 200 W.
The jump from the 4060 Ti's 128-bit to the 4070's 192-bit is where Ada starts earning its keep: 504 GB/s is ~75% more than 288, and token generation scales with it — ~55–72 t/s on 8B, ~35–46 t/s on 13B Q4. Prompt processing (600–800 t/s on 13B) is the same story. This is the first 40-series card where the speed and the VRAM stop pulling in opposite directions.
12 GB is the 13B tier's home: Q4_K_M at 32K+ with room, Q5_K_M at 8K, 8B at Q8 with a big context. It is not the 27B tier's — 19 GB of Q4 weights is 7 GB past the lid — and not the MoE 30B tier's at full quant either. The ceiling is real; the card inside it is a solid mid-range.
Between the 4060's 115 W and the 4070 Ti's 285 W, the 4070 is the efficiency midpoint of the series — a fine 24/7 card with an easy 650 W PSU requirement.
The MTP question: if your model ships a built-in multi-token-prediction head
(Qwen 3.5/3.6/3.8, Gemma 4), the one-flag speculative pass
(--spec-type draft-mtp --spec-draft-n-max 2 --parallel 1) is worth
+40–65% (estimated) on token generation here; no 4070 A/B in the record; mid-bandwidth Ada cards land in the middle of the measured spread at n-max 2. The price is ~0.6–2 GB of VRAM, the
output is bit-identical to the unaccelerated run, and models without MTP heads (Llama class) see
nothing. Full tuning rules in the MTP guide.
| Model | Size on GPU | Token gen | Token gen (MTP est.) | Prompt proc. | Fits? |
|---|---|---|---|---|---|
| 8B Q8_0 | ~9.7 GB | ~55–72 t/s | ~77–119 t/s | ~600–800 t/s | ✅ 32K ctx |
| 13B Q4_K_M | ~8.5 GB | ~35–46 t/s | ~49–76 t/s | ~650–850 t/s | ✅ 32K+ ctx |
| 13B Q5_K_M | ~11.9 GB | ~25–33 t/s | ~35–54 t/s | ~550–700 t/s | ⚠️ 8K ctx |
| 27B Q4_K_M | ~19 GB | — | — | — | ❌ ~7 GB over |
MTP est. = rough prior with --spec-type draft-mtp for models that ship MTP heads (Qwen 3.5/3.6/3.8, Gemma 4); Llama-class models get no MTP speedup. Actual gains depend on model, quant, context length, llama.cpp build and your card — sweep --spec-draft-n-max, don't trust the column blindly. MTP guide →
Figures assume full offload (-ngl 99), ~8K context and batch 2048. Token speed tracks the memory bus almost linearly, so treat ranges as class estimates — your llama.cpp build and the model's attention layout shift them.
| Quant | GGUF size | 1× 4070 (12 GB) | 2× 4070 (24 GB) |
|---|---|---|---|
| Q4_K_M | ~19 GB | ❌ ~7 GB over | ⚠️ ~16–20 t/s, 8K ctx |
| Q5_K_M | ~22 GB | ❌ | ❌ ~2 GB over |
| Q6_K | ~25 GB | ❌ | ❌ |
| Q8_0 | ~33 GB | ❌ | ❌ |
Single-card, 27B is a hard no. A pair (24 GB) gets Q4_K_M on at 8K context — workable, unhurried. Three cards (36 GB) is the proper 4070 27B build: Q5 at 8–16K around ~14–18 t/s.
-c at 8192 or the load fails.PCIe 4.0, ~25–35% tax, 12 GB tiles: 2× reaches the 27B Q4 line at 24 GB, 3× reaches Q5 at 36 GB.
| Build | GPU power | VRAM | 13B Q8 | 27B Q4_K_M | 27B Q5_K_M |
|---|---|---|---|---|---|
| 2× 4070 | 400 W | 24 GB | ~40–52 t/s | ⚠️ ~16–20 t/s, 8K ctx | ❌ ~2 GB over |
| 3× 4070 | 600 W | 36 GB | (overkill) | ~19–24 t/s, 16K+ ctx | ~14–18 t/s, 8–16K ctx |
The 3-way build is where 27B Q5 with a real context becomes a 4070 story. It wants a board with three working x8/x16 lanes — common in X670E/WRX80, uncommon elsewhere.
When a model won't fit, -ngl N parks the tail layers on the CPU. The mental model
that saves pain: offloaded layers don't run at a fraction of GPU speed — they run at CPU
speed. Handy moves: peel the last 2–4 layers off to reclaim ~0.5–1 GB of VRAM for almost
nothing; keep MoE experts on CPU with --n-cpu-moe; park the output embedding with
-ot output=CPU on big models. Budget 32 GB system RAM for 27B-class offload, 64 GB for
70B. And never offload the KV cache — the session becomes unusable.
llama-server -m 27b-q5_k_m.gguf -ngl 99 -sm layer -c 16384 --tensor-split 1,1,1
llama-server -m llama-3.1-13b-q4_k_m.gguf -ngl 99 -c 32768 -b 2048 -ub 2048 --host 127.0.0.1 --port 8080
The 4070 is the safe answer to 'which 40-series card runs 13B well': 504 GB/s, 12 GB, 200 W, and a 32K context at full quality without a knob turned. It has no drama and no ceiling surprise until 27B arrives — at which point it's a multi-card project. For the 13B lane, it's the series' default for a reason.