12 GB of GDDR7 at 672 GB/s — the mid-range finally has real bandwidth. 13B at full quality with room; 27B stays out of reach.
The 192-bit GDDR7 bus puts the 5070 ~50% ahead of the 4070's 504 GB/s and 50% ahead of the 3060's 360. Token generation follows: ~75–100 t/s on 8B, ~40–55 t/s on 13B Q4 — a pace where the quality sweet spot feels responsive. Prompt processing (~700–1000 t/s on 13B) is solid GB205-scale.
12 GB sits between the 16 GB cards and the 8 GB ones, and it lands on a convenient line: 13B Q4_K_M at 32K+ context is the comfort zone; 13B Q5_K_M (~11.9 GB) loads at 8K; 8B runs at Q8 with a big context. But 27B (~19 GB at Q4) is ~7 GB over — this is not a 27B card, and the honest path is the 5070 Ti's 16 GB or a 2-card build.
6144 cores at 250 W. Prompt processing is where this card beats the 16 GB 5060 Ti by a clear margin; token generation is where it loses (672 vs 448 GB/s). For chat, both dimensions matter — this card is the faster of the two, the 5060 Ti is the roomier one.
Before the numbers: if your model ships a built-in multi-token-prediction head
(e.g. Qwen 3.5/3.6/3.8, Gemma 4), generation can be made faster with
--spec-type draft-mtp --spec-draft-n-max 2 --parallel 1 — on this card that's worth
+40–70% (estimated) on token generation; no per-card community A/B exists for this card yet; the estimate follows the community record, where mid-bandwidth cards gain most at n-max 2. It costs ~0.6–2 GB extra VRAM and
does not change output quality, and it only applies to MTP-capable models — Llama-class
models are unaffected. Details and the tuning rules: MTP guide →
| Model | Size on GPU | Token gen | Token gen (MTP est.) | Prompt proc. | Fits? |
|---|---|---|---|---|---|
| 8B Q8_0 | ~9.7 GB | ~75–100 t/s | ~105–170 t/s | ~700–900 t/s | ✅ 32K ctx |
| 13B Q4_K_M | ~8.5 GB | ~40–55 t/s | ~57–93 t/s | ~750–1000 t/s | ✅ 32K+ ctx |
| 13B Q5_K_M | ~11.9 GB | ~28–38 t/s | ~40–64 t/s | ~600–800 t/s | ⚠️ 8K ctx |
| 27B Q4_K_M | ~19 GB | — | — | — | ❌ ~7 GB over |
MTP est. = rough prior with --spec-type draft-mtp for models that ship MTP heads (Qwen 3.5/3.6/3.8, Gemma 4); Llama-class models get no MTP speedup. Actual gains depend on model, quant, context length, llama.cpp build and your card — sweep --spec-draft-n-max, don't trust the column blindly. MTP guide →
Estimates for full GPU offload (-ngl 99), ~8K context, batch 2048. Token generation scales roughly with memory bandwidth; individual runs vary by model architecture (GQA vs MHA, MoE) and llama.cpp build.
| Quant | GGUF size | 1× 5070 (12 GB) | 2× 5070 (24 GB) |
|---|---|---|---|
| Q4_K_M | ~19 GB | ❌ ~7 GB over | ⚠️ ~22–28 t/s, 8K ctx |
| Q5_K_M | ~22 GB | ❌ | ❌ ~2 GB over |
| Q6_K | ~25 GB | ❌ | ❌ |
| Q8_0 | ~33 GB | ❌ | ❌ |
27B is out of reach single-card. A pair (24 GB) makes Q4_K_M fit at 8K context with modest speed; 3× (36 GB) is the proper build — Q5 at 8–16K around ~20–25 t/s. If 27B is the goal on a single card, the 5070 Ti's 16 GB is the minimum, not the 5070's 12.
-c at 8192 or load fails.--cache-type-k q8_0 --cache-type-v q8_0) if you push context past 32K on 13B.No NVLink: tensor split over PCIe 5.0 with a ~20–30% tax. 12 GB per tile: 2× reaches the 27B Q4 line at 24 GB, 3× reaches Q5 at 36 GB.
| Build | GPU power | VRAM | 13B Q8 | 27B Q4_K_M | 27B Q5_K_M |
|---|---|---|---|---|---|
| 2× 5070 | 500 W | 24 GB | ~45–58 t/s | ⚠️ ~22–28 t/s, 8K ctx | ❌ ~2 GB over |
| 3× 5070 | 750 W | 36 GB | (overkill) | ~26–34 t/s, 16K+ ctx | ~20–25 t/s, 8–16K ctx |
3× is the interesting point: 36 GB runs 27B at Q5 with a real context. It needs a board with three working x8/x16 slots — most X670E/WRX80 boards qualify; consumer B-series boards usually don't.
When a model doesn't fit, -ngl N keeps the last layers on the CPU instead.
The rule to internalize: offloaded layers run at CPU speed, not a percentage of GPU
speed. Practical patterns: drop just the last 2–4 layers to the CPU to reclaim ~0.5–1 GB
of VRAM (nearly free); keep MoE experts on CPU with --n-cpu-moe; park the output
embedding with -ot output=CPU to save another ~0.5–1 GB on big models. Budget 32 GB
of system RAM for 27B-class offload, 64 GB for 70B. Never offload the KV cache — the conversation
gets unusably slow.
llama-server -m 27b-q5_k_m.gguf -ngl 99 -sm layer -c 16384 --tensor-split 1,1,1
llama-server -m llama-3.1-13b-q4_k_m.gguf -ngl 99 -c 32768 -b 2048 -ub 2048 --host 127.0.0.1 --port 8080
A fast, well-balanced 13B machine — 32K context at full quality with headroom, and prompt processing that keeps long sessions feeling instant. Its weakness is the VRAM: 12 GB blocks the 27B/30B tier entirely. If the price gap to the 5070 Ti is small, the 16 GB and 896 GB/s version is the better LLM card; if it isn't, the 5070 is a comfortable daily driver for its class.