NVIDIA GTX 1070 — 344 models profiled · V1
GPU workstation Catalog
This observation used an NVIDIA GTX 1070 8GB GPU configuration. This observation profiled every Ollama model that fits in 8GB VRAM. 722 successful observations, mean 53.8 tok/s, peak 279 tok/s. CUDA, Compute 6.1.
- mean tok/s
- 53.8
- median tok/s
- 40.7
- max tok/s
- 279.0
- min tok/s
- 0.9
Four things the GPU catalog showed
A consumer-grade NVIDIA GTX 1070 8GB on a GPU workstation motherboard runs over 344 distinct Ollama models. The headline numbers below summarize what this inference rig delivers across small, mid, and saturating model sizes.
Speed king
qwen:0.5b at 230 tok/s
The Qwen 0.5B family tops the chart at 230 tok/s. Tiny models stream so fast the GPU is rarely the bottleneck — disk and network dominate.
Vision sweet spot
moondream at 190 tok/s
Moondream's vision-language family clears 190 tok/s at every variant tested — the highest sustained throughput we've seen on a vision model at this price point.
Family bands
Predictable by family
Throughput clusters tightly by family: deepseek-coder 158-161, qwen2 126-127, codegemma 117-119, falcon3 104-105. Once you pick a family, performance is predictable.
Above 3B
Throughput halves
Llama3.2 and Cogito 3B drop to 68 tok/s — half of the 1.5B/2B sweet spot. The 3B cliff is where GPU advantage starts to compress against CPU.
Models observed at highest speed on a GTX 1070 GPU
Top 10 by raw tok/s. All sub-2B models. The pattern is clear: small models run extremely fast on a consumer GPU — the qwen:0.5b family hits 230 tok/s, the moondream vision-language family hits 190+ tok/s. Above 2B, throughput drops below 100 tok/s.
- qwen:0.5b-64k
- 230.6
- qwen:0.5b-128k
- 230.5
- moondream:latest-128k
- 193.3
- moondream:latest
- 193.3
- moondream:1.8b
- 192.1
- moondream:latest-64k
- 191.9
- moondream:latest-8k
- 190.1
- qwen2.5:0.5b-128k
- 189.1
- qwen2.5:0.5b
- 188.5
- qwen2.5:0.5b-64k
- 187.7
10 entries · ordered by GPU tok/s as observed in this run · not a ranking
What each family delivers on a GTX 1070
| Family | tok/s | VRAM MB |
|---|---|---|
| deepseek-coder | 158–161 | 1,659–3,971 |
| qwen2:1.5b | 126–127 | ~2,500 |
| codegemma:2b | 117–119 | 2,241 |
| falcon3:1b | 104–105 | ~1,500 |
| gemma:2b | 91 | ~2,200 |
| smollm2 | 88–89 | ~1,200 |
| phi family | 78–80 | ~3,500 |
| gemma2:2b | 76–77 | ~2,200 |
| llama3.2 / cogito:3b | 68 | ~3,500 |