The most complete single card below 24 GB: 12 GB of GDDR6X at 960 GB/s. 13B models run fast and with real context headroom.
19 Gbps GDDR6X on the full 384-bit bus puts the 3080 Ti slightly ahead of the 3090's 936 GB/s for token generation. A 13B Q4_K_M streams at ~38–42 t/s — the single-card 13B experience in the lineup is as good as it gets below the 3090s. 8B models hit ~70 t/s: you'll be waiting for the model less often than for your own thoughts.
The 3080 10GB had to ration its KV cache; the Ti gives 13B Q4_K_M (~8.5 GB) a comfortable 3+ GB for context, meaning a real 8K–16K window, and 8B models can run at Q6/Q8 with room for 32K. It's the same "13B specialist" niche as the 3060 12GB — just at ~2× the speed and with faster prompt processing.
10240 cores at ~1.67 GHz: ~1400–1600 t/s prompt processing on 13B. Long-context ingestion is quick, and this is also the strongest sub-3090 card for large batch sizes (e.g. generating many completions in parallel in llama-server).
Before the numbers: if your model ships a built-in multi-token-prediction head
(e.g. Qwen 3.5/3.6/3.8, Gemma 4), generation can be made faster with
--spec-type draft-mtp --spec-draft-n-max 2 --parallel 1 — on this card that's worth
+45–65% (estimated) on token generation; the estimate follows the measured 3090 Ti number (42.0 → 60.9 t/s at n-max 2). It costs ~0.6–2 GB extra VRAM and
does not change output quality, and it only applies to MTP-capable models — Llama-class
models are unaffected. Details and the tuning rules: MTP guide →
| Model | Size on GPU | Token gen | Token gen (MTP est.) | Prompt proc. | Fits? |
|---|---|---|---|---|---|
| 8B (Llama 3.1 8B, Mistral) | ~4.9 GB | ~99–130 t/s | ~144–214 t/s | ~1200–1500 t/s | ✅ Q8_0 with big ctx |
| 13B Q4_K_M | ~8.5 GB | ~58–73 t/s | ~84–120 t/s | ~800–1000 t/s | ✅ 8K–16K ctx |
| 13B Q6_K | ~10.6 GB | ~47–61 t/s | ~69–101 t/s | ~750 t/s | ✅ 4K–8K ctx |
| 20B+ dense | ~13 GB | — | — | — | ❌ just over |
MTP est. = rough prior with --spec-type draft-mtp for models that ship MTP heads (Qwen 3.5/3.6/3.8, Gemma 4); Llama-class models get no MTP speedup. Actual gains depend on model, quant, context length, llama.cpp build and your card — sweep --spec-draft-n-max, don't trust the column blindly. MTP guide →
Estimates, full GPU offload, ~8K context.
| Quant | GGUF size | 1× 3080 Ti (12 GB) | 2× 3080 Ti (24 GB) |
|---|---|---|---|
| Q4_K_M | ~19 GB | ❌ ~7 GB over | ✅ ~26–30 t/s, 8–16K ctx |
| Q5_K_M | ~22 GB | ❌ | ⚠️ ~20–22 t/s, 4–8K ctx |
| Q6_K | ~25 GB | ❌ | ❌ ~1 GB over |
| Q8_0 | ~33 GB | ❌ | ❌ |
The single card is 12 GB: 27B doesn't fit, same wall as the 3060 12GB (but that pair runs the same 27B Q4 at 2× the speed). The dual build is the fastest 24 GB 27B rig below the 3090s: ~26 t/s on Q4_K_M is a genuinely pleasant chat speed, and Q5_K_M is usable with a short context. Q6/Q8 want 36 GB+, which is a 3090 mixed-build conversation (3090 + 3080 Ti = 36 GB, both quants fit).
-c 16384 is the sweet spot; ~40 t/s with a real context window.Single-card numbers above. 960 GB/s per tile makes this the fastest PCIe 30-series tile — no NVLink, so the ~30% split tax still applies.
| Build | GPU power | VRAM | 13B Q4 | 27B Q4_K_M | 27B Q5_K_M | 27B Q6_K | 27B Q8_0 |
|---|---|---|---|---|---|---|---|
| 2× 3080 Ti | 700 W | 24 GB | ~65 t/s | ✅ ~25–28 t/s | ⚠️ ~20–22 t/s, 4–8K | ❌ ~1 GB over | ❌ |
| 3× 3080 Ti | 1050 W | 36 GB | ~78 t/s | ~33 t/s | ~28 t/s | ~25–27 t/s | ⚠️ ~18–20 t/s, 8K |
| 4× 3080 Ti | 1400 W | 48 GB | ~100 t/s | ~38 t/s | ~33 t/s | ~30–32 t/s | ~23–25 t/s |
| 4× 3080 Ti · 70B | 1400 W | 48 GB | — | 70B Q4_K_M at ~20–23 t/s — strong, but 2× 3090 with NVLink beats it for 200 W less | |||
3× (36 GB) is the 27B-Q6 point for a PCIe-only money: ~26 t/s, a quant that a 24 GB box can't even hold. 4× (48 GB) is a full 70B-Q4 / 27B-Q8 rig — the strongest all-PCIe 48 GB build in the RTX 30 Series, at the price of 1.4 kW and a workstation board. If your budget touches 3090 prices, the mixed 3090+3080 Ti (36 GB) and 2× 3090 (48 GB) builds leapfrog the 3×/4× Tis on power and interconnect.
When a model doesn't fit, -ngl N keeps the last layers on the CPU instead.
The rule to internalize: offloaded layers run at CPU speed, not a percentage of GPU
speed. A 27B model that's half offloaded is not 50% of GPU speed — it's ~3–5 t/s
total on a modern 8-core. Practical patterns: drop just the last 2–4 layers to the CPU to
reclaim ~0.5–1 GB of VRAM (nearly free); keep MoE experts on CPU with --n-cpu-moe;
park the output embedding with -ot output=CPU to save another ~0.5 GB on big
models. Budget 32 GB of system RAM for 27B-class offload, 64 GB for 70B. Never offload the
KV cache — the conversation gets unusably slow. Offload is only interesting on the 24 GB pair, for the 1-GB Q6 gap: the IQ6_XS imatrix quant (~24 GB) fits without offloading and is the better fix.
llama-server -m 27b-q6_k.gguf -ngl 99 -sm layer -c 16384 --tensor-split 1,1,1llama-server -m 70b-q4_k_m.gguf -ngl 99 -sm layer -c 8192 --tensor-split 1,1,1,1
llama-server -m llama-3-13b-q4_k_m.gguf -ngl 99 -c 16384 -b 2048 -ub 2048 --host 127.0.0.1 --port 8080
The best single-card 13B experience under 24 GB: near-3090 bandwidth, honest 12 GB, and strong prompt processing. It loses to the 3090 on capacity (and price, if you can find either), but as a fast 8–13B machine — or a tile in a mixed multi-GPU build — the 3080 Ti has no equal below the 3090 line.