Best hardware for Gemma 4 12B
Light assistant for mid-range GPUs.
12B parameters · dense · gemma family
Our picks for this model
Cheapest · Easiest
1× NVIDIA GeForce RTX 3060 12GB
$340 on eBay ↗ + shipping unknown · ~23.5 tok/s est.
one card, no extra setup
Rule-based picks at Q4_K_M, from live screened prices (how we estimate speed and choose picks). Every option is ranked below.
Q4_K_M
Minimum VRAM: 10.05 GB (file 7.66 GB + context/runtime headroom at 8k ctx; see methodology).
Cheapest fitting cloud offer: Vast.ai $0.109/hr (rtx-a4000 class, on-demand) · cloud prices checked 2026-10-29.
| Config | Screened price | Total VRAM | Est. watts | Est. tokens/sec | Breakeven vs cloud | Verdict |
|---|---|---|---|---|---|---|
| 1× NVIDIA GeForce RTX 3060 12GB our pick · Cheapest · Easiest | $340 + shipping unknown | 12 GB | 245 W (well below this table's average) | ~23.5 (well below this table's average) | 36 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 5090 our pick · Fastest | $7,070 | 32 GB | 650 W | ~117 (well above this table's average) | 11,808 mo | Renting wins at 40 h/mo |
| 1× Intel Arc B580 Intel card: more setup than NVIDIA for local AI
| $389 | 12 GB | 265 W (well below this table's average) | ~29.8 | 50 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 3080 10GB
fits with a small context window (about 4k)
| $443 | 10 GB | 395 W | ~49.6 | 65 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 3080 12GB | $530 | 12 GB | 425 W | ~59.6 | 86 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 4060 Ti 16GB | $720 | 16 GB | 240 W (well below this table's average) | ~18.8 (well below this table's average) | 87 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 5060 Ti 16GB | $838 | 16 GB | 255 W (well below this table's average) | ~29.2 (well below this table's average) | 129 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 4070 Ti SUPER | $975 | 16 GB | 360 W | ~43.9 | 127 mo | Renting wins at 40 h/mo |
| 1× NVIDIA RTX A4000 | $980 | 16 GB | 215 W (well below this table's average) | ~29.2 (well below this table's average) | 91 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 4080 | $1,100 | 16 GB | 395 W | ~46.8 | 154 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 4080 SUPER | $1,240 | 16 GB | 395 W | ~48.1 | 216 mo | Renting wins at 40 h/mo |
| 1× AMD Radeon RX 7900 XTX AMD card: more setup than NVIDIA for local AI
| $1,280 | 24 GB | 430 W | ~62.7 | 247 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 5070 Ti | $1,350 | 16 GB | 375 W | ~58.5 | 228 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 3090 | $1,500 | 24 GB | 425 W | ~61.1 | 208 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 3090 Ti | $1,810 | 24 GB | 525 W | ~65.8 | 424 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 5080 | $2,050 | 16 GB | 435 W | ~62.7 | 477 mo | Renting wins at 40 h/mo |
| 1× NVIDIA RTX A5000 | $2,349 | 24 GB | 305 W (well below this table's average) | ~50.1 | 331 mo | Renting wins at 40 h/mo |
| 2× NVIDIA GeForce RTX 3090 | $3,070 | 48 GB | 775 W (well above this table's average) | ~61.1 | never | Cloud always cheaper at this power price |
| 1× NVIDIA GeForce RTX 4090 | $3,720 | 24 GB | 525 W | ~65.8 | 1,433 mo | Renting wins at 40 h/mo |
| 1× NVIDIA RTX A6000 | $4,460 | 48 GB | 375 W | ~50.1 | 619 mo | Renting wins at 40 h/mo |
| 2× NVIDIA RTX A5000 | $4,729 | 48 GB | 535 W | ~50.1 | 1,741 mo | Renting wins at 40 h/mo |
| 4× NVIDIA GeForce RTX 3090 | $6,269 | 96 GB | 1475 W (well above this table's average) | ~61.1 | never | Cloud always cheaper at this power price |
| 2× NVIDIA GeForce RTX 4090 | $7,520 | 48 GB | 975 W (well above this table's average) | ~65.8 | never | Cloud always cheaper at this power price |
| 2× NVIDIA RTX A6000 | $8,930 | 96 GB | 675 W | ~50.1 | 68,344 mo | Renting wins at 40 h/mo |
| 4× NVIDIA RTX A5000 | $9,529 | 96 GB | 995 W (well above this table's average) | ~50.1 | never | Cloud always cheaper at this power price |
| 1× NVIDIA Tesla P40 Passive datacenter card: needs a cooling mod and is far slower than modern cards | $350 | 24 GB | 325 W | ~22.7 (well below this table's average) | 38 mo | Renting wins at 40 h/mo |
| 2× NVIDIA Tesla P40 Passive datacenter card: needs a cooling mod and is far slower than modern cards | $700 + shipping unknown on 1 of 2 | 48 GB | 575 W | ~22.7 (well below this table's average) | 258 mo | Renting wins at 40 h/mo |
| 1× NVIDIA TITAN RTX Turing-era card: much slower than an RTX 3090 for similar money | $1,168 | 24 GB | 355 W | ~43.9 | 184 mo | Renting wins at 40 h/mo |
| 4× NVIDIA Tesla P40 Passive datacenter card: needs a cooling mod and is far slower than modern cards | $1,426 + shipping unknown on 1 of 4 | 96 GB | 1075 W (well above this table's average) | ~22.7 (well below this table's average) | never | Cloud always cheaper at this power price |
- Our pick rows sit on top, then clean configs by price; cards with a caveat (cooling mod needed, or poor value for AI) group last, whatever their price. Watts and tokens/sec turn green or red when they sit far from this table's average (roughly 40% either way).
- Prices link straight to the screened eBay listing; hover for the seller summary. Seller details and the report option live on each card's own page.
- Configs without a screened purchase option are hidden until listings clear the trust screen (a multi-day survival window on eBay).
- Multi-card costs are the sum of that many separate screened listings; the cheapest listing counted twice is not a price anyone can pay. Prices exclude the PSU, board and case a multi-card build usually also needs.
- Tokens/sec are estimates, not benchmarks: spec-sheet memory bandwidth at 50% utilization divided by the model bytes read per token (how and why). Real speeds vary with the runtime and context length.
- Breakeven and verdict use default assumptions (40 h/mo, $0.16/kWh, 24-month horizon, resale at 65% of the going rate); adjust them in the calculator. Hardware links are screened, not guaranteed.
Run your own numbers for Gemma 4 12B (Q4_K_M) in the buy-vs-rent calculator →
Q8_0
Minimum VRAM: 15.31 GB (file 12.67 GB + context/runtime headroom at 8k ctx; see methodology).
Cheapest fitting cloud offer: Vast.ai $0.109/hr (rtx-a4000 class, on-demand) · cloud prices checked 2026-10-29.
| Config | Screened price | Total VRAM | Est. watts | Est. tokens/sec | Breakeven vs cloud | Verdict |
|---|---|---|---|---|---|---|
| 1× NVIDIA GeForce RTX 4060 Ti 16GB | $720 | 16 GB | 240 W (well below this table's average) | ~11.4 (well below this table's average) | 87 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 5060 Ti 16GB | $838 | 16 GB | 255 W (well below this table's average) | ~17.7 (well below this table's average) | 129 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 4070 Ti SUPER | $975 | 16 GB | 360 W | ~26.5 | 127 mo | Renting wins at 40 h/mo |
| 1× NVIDIA RTX A4000 | $980 | 16 GB | 215 W (well below this table's average) | ~17.7 (well below this table's average) | 91 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 4080 | $1,100 | 16 GB | 395 W | ~28.3 | 154 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 4080 SUPER | $1,240 | 16 GB | 395 W | ~29.1 | 216 mo | Renting wins at 40 h/mo |
| 1× AMD Radeon RX 7900 XTX AMD card: more setup than NVIDIA for local AI
| $1,280 | 24 GB | 430 W | ~37.9 | 247 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 5070 Ti | $1,350 | 16 GB | 375 W | ~35.4 | 228 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 3090 | $1,500 | 24 GB | 425 W | ~36.9 | 208 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 3090 Ti | $1,810 | 24 GB | 525 W | ~39.8 | 424 mo | Renting wins at 40 h/mo |
| 1× NVIDIA GeForce RTX 5080 | $2,050 | 16 GB | 435 W | ~37.9 | 477 mo | Renting wins at 40 h/mo |
| 1× NVIDIA RTX A5000 | $2,349 | 24 GB | 305 W (well below this table's average) | ~30.3 | 331 mo | Renting wins at 40 h/mo |
| 2× NVIDIA GeForce RTX 3090 | $3,070 | 48 GB | 775 W (well above this table's average) | ~36.9 | never | Cloud always cheaper at this power price |
| 1× NVIDIA GeForce RTX 4090 | $3,720 | 24 GB | 525 W | ~39.8 | 1,433 mo | Renting wins at 40 h/mo |
| 1× NVIDIA RTX A6000 | $4,460 | 48 GB | 375 W | ~30.3 | 619 mo | Renting wins at 40 h/mo |
| 2× NVIDIA RTX A5000 | $4,729 | 48 GB | 535 W | ~30.3 | 1,741 mo | Renting wins at 40 h/mo |
| 4× NVIDIA GeForce RTX 3090 | $6,269 | 96 GB | 1475 W (well above this table's average) | ~36.9 | never | Cloud always cheaper at this power price |
| 1× NVIDIA GeForce RTX 5090 | $7,070 | 32 GB | 650 W | ~70.7 (well above this table's average) | 11,808 mo | Renting wins at 40 h/mo |
| 2× NVIDIA GeForce RTX 4090 | $7,520 | 48 GB | 975 W (well above this table's average) | ~39.8 | never | Cloud always cheaper at this power price |
| 2× NVIDIA RTX A6000 | $8,930 | 96 GB | 675 W | ~30.3 | 68,344 mo | Renting wins at 40 h/mo |
| 4× NVIDIA RTX A5000 | $9,529 | 96 GB | 995 W (well above this table's average) | ~30.3 | never | Cloud always cheaper at this power price |
| 1× NVIDIA Tesla P40 Passive datacenter card: needs a cooling mod and is far slower than modern cards | $350 | 24 GB | 325 W (well below this table's average) | ~13.7 (well below this table's average) | 38 mo | Renting wins at 40 h/mo |
| 2× NVIDIA Tesla P40 Passive datacenter card: needs a cooling mod and is far slower than modern cards | $700 + shipping unknown on 1 of 2 | 48 GB | 575 W | ~13.7 (well below this table's average) | 258 mo | Renting wins at 40 h/mo |
| 1× NVIDIA TITAN RTX Turing-era card: much slower than an RTX 3090 for similar money | $1,168 | 24 GB | 355 W | ~26.5 | 184 mo | Renting wins at 40 h/mo |
| 4× NVIDIA Tesla P40 Passive datacenter card: needs a cooling mod and is far slower than modern cards | $1,426 + shipping unknown on 1 of 4 | 96 GB | 1075 W (well above this table's average) | ~13.7 (well below this table's average) | never | Cloud always cheaper at this power price |
- Our pick rows sit on top, then clean configs by price; cards with a caveat (cooling mod needed, or poor value for AI) group last, whatever their price. Watts and tokens/sec turn green or red when they sit far from this table's average (roughly 40% either way).
- Prices link straight to the screened eBay listing; hover for the seller summary. Seller details and the report option live on each card's own page.
- Configs without a screened purchase option are hidden until listings clear the trust screen (a multi-day survival window on eBay).
- Multi-card costs are the sum of that many separate screened listings; the cheapest listing counted twice is not a price anyone can pay. Prices exclude the PSU, board and case a multi-card build usually also needs.
- Tokens/sec are estimates, not benchmarks: spec-sheet memory bandwidth at 50% utilization divided by the model bytes read per token (how and why). Real speeds vary with the runtime and context length.
- Breakeven and verdict use default assumptions (40 h/mo, $0.16/kWh, 24-month horizon, resale at 65% of the going rate); adjust them in the calculator. Hardware links are screened, not guaranteed.
Run your own numbers for Gemma 4 12B (Q8_0) in the buy-vs-rent calculator →