Xeon vs AMD workstation — CPU inference head-to-head · V1

Cross-machine CPU inference

Does a bigger CPU make local AI meaningfully faster? This is a CPU-to-CPU comparison with no GPU on either side. Sigma5C observed 71 small models with the same three prompts on two CPUs: a dual Intel Xeon E5-2690 v3 (48 threads, 157 GB RAM) and an AMD workstation (Athlon 3000G, 4 threads, 60 GB RAM). The larger CPU was about 1.8 times faster on average, far less than its core advantage, which is consistent with memory bandwidth rather than core count limiting CPU inference in this run.

mean Xeon advantage: 1.8×
max (phi:latest): 3.0×
median: 1.7×
AMD workstation lead (smollm2): 1

What is being compared

Xeon server CPU

2× Intel Xeon E5-2690 v3 · 48 threads · 157 GB RAM

AMD workstation

AMD Athlon 3000G · 2 cores / 4 threads · 60 GB RAM

GPU: NVIDIA GTX 1070, 8 GB (not used in this CPU-only run).

Where the Xeon leads most, and the one model where the AMD workstation leads

Xeon tok/s (sample)

granite3-moe:latest: 87.9
granite3-moe: 87.9
functiongemma:270m: 73.5
qwen2.5:0.5b: 65.3
smollm2:135m: 50.5
phi:latest: 12.5

6 entries · ordered by Xeon tok/s (sample) as observed in this run · not a ranking

CPU inference observations with 48 threads

Mean speedup ~1.8×

Less than the 24× core ratio

The 48-thread Xeon does not scale linearly. Memory bandwidth is the real bottleneck for CPU inference; the 2-core Athlon has identical DDR4 bandwidth per channel.

Largest model wins

Older Phi grew the gap

phi:latest ran 3.0× faster on the Xeon (12.5 vs 4.1 tok/s). Older quantizations exhibit larger speed gradients across core counts.

Tiny models converge

smollm2:1.7b is 20% faster on the AMD workstation

One model — and it's the tiniest — ran faster on the smaller machine. Memory access patterns at very-small parameter counts make extra cores irrelevant or counterproductive (more context switching).

The AMD workstation is the GPU candidate

Why we observed this CPU baseline first

The AMD workstation carries an NVIDIA GTX 1070 (8 GB) and was selected for the GPU observations. Establishing the CPU-only baseline first gives a clean control: any later speedup can be attributed to GPU acceleration rather than to more cores.