NVIDIA GeForce RTX 30 Series

Pick your card

Each page covers the card's specs, what those specs mean for llama.cpp, what models fit, expected speeds, and how it behaves in multi-GPU builds.

What makes this series different

• GDDR6X split. The 3070 Ti and the 3080 class moved to GDDR6X and the bandwidth story jumped: 912 GB/s on the 3070 Ti, 760 on the 3080 10G, 912 on the 3080 12G, 960 on the 3080 Ti, 936 on the 3090. Below that, the 3050–3070 share one 128/192/256-bit GDDR6 envelope and differ mostly in compute.
• NVLink lives and dies here. The 3090/3090 Ti (112.5 GB/s) are the only consumer cards with a fast GPU-to-GPU bridge. Every serious 70B build on this site is a 3090 build because of it — nothing in the RTX 40 or RTX 50 Series replaced it.
• The value tier is weirdly good. The 3060 12 GB still anchors the budget multi-GPU picks (2×/3×/4× for 24/36/48 GB) because it's cheap, cool and 12 GB per tile.

Single vs. multiple cards

llama.cpp splits a model's transformer layers across GPUs (tensor split). Two rules of thumb:

• Token generation scales well — bandwidths roughly add, so 2× 3090 ≈ double the t/s of one, and you get 48 GB of VRAM (70B Q4 territory).
• Inter-GPU traffic is the tax. Layers on different cards exchange activations every token. Over PCIe 4.0 that's fine-ish; the 3090's NVLink makes 2-GPU splits much cheaper. Cards without NVLink (3050–3080 Ti) can still be combined, with a larger penalty.
• 3-way and 4-way builds exist for boards that support them. 3× 3060 12G (36 GB) is the budget 27B-Q6 point; 3× 3090 (72 GB) is the long-context 70B point; 4× tops out at 96 GB on the 3090 line. Note: 3090 NVLink is 2-way only, so extra cards join over PCIe.

Each card page has a Multi-GPU section with the exact llama-cli / llama-server flags for that card. The series comparison has scaling tables for common 2-GPU and 4-GPU builds, and the multi-GPU guide covers the mechanics.