A consumer GPU recorded higher throughput than a 48-core Xeon · V1
GPU vs CPU Crossover
The most consequential single finding from the GPU workstation upgrade. A NVIDIA GTX 1070 8GB on an AMD GPU workstation board recorded 5.7× higher throughput than a 48-thread Xeon server CPU (157 GB RAM) on average — for any model that fits in 5 GB of VRAM. The crossover is clean: GPU recorded higher when the model fits, CPU when it doesn't.
- avg GPU vs Xeon CPU
- 5.7×
- GPU workstation vs GPU workstation own CPU
- 9.0×
- peak band (2-4GB VRAM)
- 7.2×
- GPU
- GTX 1070
Where the GPU advantage dies
Below 6 GB of VRAM, the GPU dominates the 48-core Xeon by 5-7×. Between 6 and 8 GB (the limit of the GTX 1070), the GPU still wins — but the margin narrows to 2.5× as the model approaches the VRAM ceiling. Above 8 GB, the model doesn't fit and CPU wins by default.
| VRAM Bucket | Models | Avg GPU tok/s | Avg Xeon CPU tok/s | Ratio | Verdict |
|---|---|---|---|---|---|
| < 2 GB | 7 | 156.6 | 27.0 | 5.8× | GPU dominates |
| 2 – 4 GB | 64 | 111.4 | 15.6 | 7.2× | GPU dominates (peak ratio) |
| 4 – 6 GB | 112 | 35.3 | 7.0 | 5.1× | GPU dominates |
| 6 – 8 GB | 66 | 14.6 | 5.7 | 2.5× | GPU throughput, narrower margin |
GPU throughput differences observed in this run
Each row gives the three measurements: GPU tok/s, Xeon CPU tok/s, GPU workstation own CPU tok/s. The ratio column is GPU / Xeon CPU — the relevant comparison for "should I buy a GPU or pay for cloud Xeon?"
- deepseek-coder:latest
- 8.3
- deepseek-coder:1.3b
- 7.1
- gemma:2b
- 7
- qwen2:1.5b
- 6.9
- codegemma:2b
- 4.9
- granite3.1-moe:latest
- 4
6 entries · ordered by GPU vs Xeon CPU as observed in this run · not a ranking
Why this changes economics
A single GTX 1070 on a GPU workstation board recorded higher throughput than a 48-core Xeon server for any model under ~5GB VRAM. The crossover point is clear: GPU recorded higher when the model fits, CPU when it didn't. In this observation, the tested GPU configuration recorded higher throughput than the tested Xeon configuration for typical agentic-coding model sizes (1B–3B).