16 GB + 896 GB/s — the sub-flagship balance. 3090-class generation speed at a smaller power budget, with 16 GB instead of 24.
The 256-bit GDDR7 bus puts the 5070 Ti within 4% of the 3090's 936 GB/s and 10% short of the 3090 Ti. Token generation lands in ~100–130 t/s on 8B, ~53–70 t/s on 13B Q4 — the speed tier the rest of this lineup chases, at 300 W instead of 350–450. Prompt processing (~900–1300 t/s on 13B) is the strongest in the sub-flagship tier.
16 GB runs 13B at Q6 and MoE 30B at Q3–Q4, and 27B at Q3 with a short context. But the 3090's 24 GB ran 27B at Q4–Q5 comfortably — the 5070 Ti is faster per token yet fits less. It's the defining trade of the sub-flagship tier: speed up, capacity down. 27B at Q4+ is a 2-card project (32 GB covers Q4/Q5/Q6).
8960 cores at 300 W — efficient by flagship standards. The card runs cooler than a 4070 Ti at the same load, which suits a 24/7 box.
Before the numbers: if your model ships a built-in multi-token-prediction head
(e.g. Qwen 3.5/3.6/3.8, Gemma 4), generation can be made faster with
--spec-type draft-mtp --spec-draft-n-max 2 --parallel 1 — on this card that's worth
+45–75% (estimated) on token generation; no per-card community A/B exists for this card yet; the estimate follows the measured 3090 Ti result (+45–47% at n-max 2) at similar bus width. It costs ~0.6–2 GB extra VRAM and
does not change output quality, and it only applies to MTP-capable models — Llama-class
models are unaffected. Details and the tuning rules: MTP guide →
| Model | Size on GPU | Token gen | Token gen (MTP est.) | Prompt proc. | Fits? |
|---|---|---|---|---|---|
| 8B Q8_0 | ~9.7 GB | ~100–130 t/s | ~145–227 t/s | ~900–1200 t/s | ✅ 32K+ ctx |
| 13B Q4_K_M | ~8.5 GB | ~53–70 t/s | ~77–122 t/s | ~1000–1300 t/s | ✅ 32K+ ctx |
| 13B Q6_K | ~13.8 GB | ~40–53 t/s | ~58–92 t/s | ~900–1200 t/s | ✅ 8–16K ctx |
| 27B Q3_K_M | ~15.8 GB | ~17–23 t/s | ~25–40 t/s | ~700–900 t/s | ⚠️ 4–8K ctx |
| MoE 30B-A3B (Q3) | ~14.5 GB | ~75–100 t/s | ~110–175 t/s | ~900–1200 t/s | ⚠️ 4–8K ctx, 3B active |
MTP est. = rough prior with --spec-type draft-mtp for models that ship MTP heads (Qwen 3.5/3.6/3.8, Gemma 4); Llama-class models get no MTP speedup. Actual gains depend on model, quant, context length, llama.cpp build and your card — sweep --spec-draft-n-max, don't trust the column blindly. MTP guide →
Estimates for full GPU offload (-ngl 99), ~8K context, batch 2048. Token generation scales roughly with memory bandwidth; individual runs vary by model architecture (GQA vs MHA, MoE) and llama.cpp build. MoE token gen depends on active params, not total — treat as a bonus speed tier.
| Quant | GGUF size | 1× 5070 Ti (16 GB) | 2× 5070 Ti (32 GB) |
|---|---|---|---|
| Q4_K_M | ~19 GB | ❌ ~3 GB over | ✅ ~38–48 t/s, 16K+ ctx |
| Q5_K_M | ~22 GB | ❌ | ✅ ~32–40 t/s, 8–16K ctx |
| Q6_K | ~25 GB | ❌ | ✅ ~28–35 t/s, 8–16K ctx |
| Q8_0 | ~33 GB | ❌ | ❌ ~1 GB over |
Single-card 27B is Q3 territory. The 2× build (600 W, 32 GB) is where the card's speed pays off: Q4_K_M at ~38–48 t/s with 16K+ context — a full 3090-pair experience at modern speeds — plus Q5 and Q6 at 8–16K. Q8 misses by a GB, the usual 16+16 arithmetic.
-c 8192, KV quantized — usable, quality-limited.No NVLink — the split runs over PCIe 5.0 with a ~20–30% tax (a 3090 pair's NVLink costs ~10–15%). 16 GB per tile: 2× reaches 32 GB, the 27B Q4–Q6 line.
| Build | GPU power | VRAM | 27B Q4_K_M | 27B Q5_K_M | 27B Q6_K |
|---|---|---|---|---|---|
| 2× 5070 Ti | 600 W | 32 GB | ~38–48 t/s, 16K+ ctx | ~32–40 t/s, 8–16K ctx | ~28–35 t/s, 8–16K ctx |
The 2-card build is the card's endgame: 32 GB at 3090-pair speeds, minus NVLink's comfort. Standard ATX with two x16/x8 slots works; a 750 W PSU and good airflow close the bill.
When a model doesn't fit, -ngl N keeps the last layers on the CPU instead.
The rule to internalize: offloaded layers run at CPU speed, not a percentage of GPU
speed. Practical patterns: drop just the last 2–4 layers to the CPU to reclaim ~0.5–1 GB
of VRAM (nearly free); keep MoE experts on CPU with --n-cpu-moe; park the output
embedding with -ot output=CPU to save another ~0.5–1 GB on big models. Budget 32 GB
of system RAM for 27B-class offload, 64 GB for 70B. Never offload the KV cache — the conversation
gets unusably slow.
llama-server -m 27b-q4_k_m.gguf -ngl 99 -sm layer -c 16384 --tensor-split 1,1
llama-server -m llama-3.1-13b-q6_k.gguf -ngl 99 -c 16384 -b 2048 -ub 2048 --host 127.0.0.1 --port 8080
The fastest sub-5090 card for token generation, and a proper 16 GB machine: 13B at Q6, MoE 30B at real speed, and a 2-card path to 27B at Q4–Q6. Its identity is the trade — 3090-class speed, 3090-minus capacity. If single-card 27B Q4 is the goal, this isn't the card; if it's fast 13B/MoE today with a 2-card option later, it's the tier's default pick.