The value king. Identical GA106 silicon to the 8GB model, but 50% more VRAM — and that's what unlocks 13B models at full quantization quality.
This is the card where the "what fits" math changes. A 13B/14B Q4_K_M model (~8.5 GB) now fits with 3+ GB left for KV cache — enough for a solid 8K–16K context. That's the entire difference between running Llama 3 8B and Llama 3.1 8B (same card) versus running 13B-class models that are meaningfully smarter. Add the Q5_K option for 8B models, and small MoE models with a few active GB. Nothing else in the lineup undercuts this card for 12 GB.
Same 192-bit GDDR6 bus as the 8GB model, so token generation speed is the same: ~35 t/s on 8B, ~20 t/s on 13B Q4. That 13B figure is "usable" rather than "delightful" — comfortable for agentic/coding loops and background work, slightly slow for chatting in a hurry. If raw 13B speed matters more than the price tag, the 3080 (760 GB/s) runs it at ~2×.
3584 cores is mid-pack, but GA106 is an efficient chip and 170 W means it boosts cleanly for long inference sessions without throttling — a real advantage over 350 W cards in multi-GPU builds where sustained load is the norm.
Before the numbers: if your model ships a built-in multi-token-prediction head
(e.g. Qwen 3.5/3.6/3.8, Gemma 4), generation can be made faster with
--spec-type draft-mtp --spec-draft-n-max 2 --parallel 1 — on this card that's worth
+40–70% (estimated) on token generation; 3× 3060 12G builds measured +42% at n-max 8 with layer split; the single-card figure is an estimate from that data. It costs ~0.6–2 GB extra VRAM and
does not change output quality, and it only applies to MTP-capable models — Llama-class
models are unaffected. Details and the tuning rules: MTP guide →
| Model | Size on GPU | Token gen | Token gen (MTP est.) | Prompt proc. | Fits? |
|---|---|---|---|---|---|
| 8B (Llama 3.1 8B, Mistral) | ~4.9 GB | ~46–64 t/s | ~64–109 t/s | ~500–600 t/s | ✅ big ctx, or Q6_K |
| 13–14B dense (Llama 3 13B, Gemma 3 12B) | ~8.5 GB | ~25–37 t/s | ~35–63 t/s | ~350 t/s | ✅ 8K–16K ctx |
| 13B Q5_K_M | ~9.6 GB | ~22–32 t/s | ~31–54 t/s | ~320 t/s | ✅ with 8K ctx |
| MoE 30B-A3B (Qwen3-30B) | ~18 GB | — | — | — | ❌ (needs ~18 GB) |
| 30B dense | ~18 GB | — | — | — | ❌ no |
MTP est. = rough prior with --spec-type draft-mtp for models that ship MTP heads (Qwen 3.5/3.6/3.8, Gemma 4); Llama-class models get no MTP speedup. Actual gains depend on model, quant, context length, llama.cpp build and your card — sweep --spec-draft-n-max, don't trust the column blindly. MTP guide →
Estimates, full GPU offload, ~8K context.
| Quant | GGUF size | 1× 3060 12G (12 GB) | 2× 3060 12G (24 GB) |
|---|---|---|---|
| Q4_K_M | ~19 GB | ❌ ~7 GB over | ✅ ~10–12 t/s, 8–16K ctx |
| Q5_K_M | ~22 GB | ❌ | ⚠️ ~9 t/s, 4–8K ctx |
| Q6_K | ~25 GB | ❌ | ❌ ~1 GB over |
| Q8_0 | ~33 GB | ❌ | ❌ |
This is the cheapest 27B build in the entire lineup: two cards = 24 GB = a 27B Q4_K_M that actually runs (~10–12 t/s, comfortably usable for background and agentic work). Q5_K_M also loads but leaves almost nothing for the KV cache, so keep context short. Q6 and Q8 require 36 GB+, which is 3–4 card territory. On a single card, 27B is only possible via partial CPU offload (≈22 of 64 layers on GPU): ~5–7 t/s on Q4 — a fine "it works" option while a second card is on order.
-c explicitly on 27B Q5 builds; auto-context will happily OOM you.Single-card numbers are in the table above. The 3060 12GB is the tile that makes multi-GPU 27B builds affordable — here's the full ladder.
| Build | GPU power | VRAM | 13B Q4 | 27B Q4_K_M | 27B Q5_K_M | 27B Q6_K | 27B Q8_0 |
|---|---|---|---|---|---|---|---|
| 2× 3060 12G | 340 W | 24 GB | ~35 t/s | ✅ ~10–12 t/s | ⚠️ ~9 t/s, 4–8K | ❌ | ❌ |
| 3× 3060 12G | 510 W | 36 GB | ~46 t/s | ~14 t/s | ~16 t/s | ~10–12 t/s | ❌ ~2 GB over |
| 4× 3060 12G | 680 W | 48 GB | ~60 t/s | ~18 t/s | ~14–16 t/s | ~12–14 t/s | ~9–10 t/s |
| 4× 3060 12G · 70B | 680 W | 48 GB | — | 70B Q4_K_M at ~9–11 t/s — the cheapest 70B in the lineup | |||
4× is the value king of this whole site: 48 GB at 3060 prices means 27B at any quant you can stand (Q6 at ~13 t/s is the sweet spot) and 70B Q4 as a stretch goal. The 3-card build (36 GB) is the "27B Q6 on a budget" point. Requirements: a board with 3–4 x8/x16 slots — X570/X670E boards often do 4-way x8 — a 600–800 W PSU, and airflow for four 170 W cards (it stays modest because the chips are efficient).
When a model doesn't fit, -ngl N keeps the last layers on the CPU instead.
The rule to internalize: offloaded layers run at CPU speed, not a percentage of GPU
speed. A 27B model that's half offloaded is not 50% of GPU speed — it's ~3–5 t/s
total on a modern 8-core. Practical patterns: drop just the last 2–4 layers to the CPU to
reclaim ~0.5–1 GB of VRAM (nearly free); keep MoE experts on CPU with --n-cpu-moe;
park the output embedding with -ot output=CPU to save another ~0.5 GB on big
models. Budget 32 GB of system RAM for 27B-class offload, 64 GB for 70B. Never offload the
KV cache — the conversation gets unusably slow. Offload is a fallback here, not a plan: 2× covers 27B Q4 fully, and only the 70B-on-4× build genuinely needs RAM to stand behind it (64 GB).
llama-server -m 27b-q6_k.gguf -ngl 99 -sm layer -c 8192 --tensor-split 1,1,1llama-server -m 27b-q8_0.gguf -ngl 99 -sm layer -c 8192 --tensor-split 1,1,1,1llama-server -m 70b-q4_k_m.gguf -ngl 99 -sm layer -c 8192 --tensor-split 1,1,1,1
llama-server -m llama-3-13b-q4_k_m.gguf -ngl 99 -c 8192 -b 2048 -ub 2048 --host 127.0.0.1 --port 8080
The RTX 3060 12GB is the default answer to "which GPU for local LLMs on a budget." It runs the standard 13B model at full quality, it's cool and cheap, and it tiles beautifully into 2× and 4× builds that reach 30B and even 70B territory. You give up speed versus the 3080/3090 and 24 GB versus a used 3090 — but you rarely give up capability at a similar price. Buy two if you can.