Function-calling — 9 cloud models × 30 prompts · V1
Tool Calling Observation
- total observations
- 270
- perfect models
- 6
- false-positive prone
- 1
- wrong-tool issue
- 1
Tool-call accuracy observed score table
- qwen3-coder:480b-cloud
- 12
- qwen3.5:cloud
- 12
- devstral-small-2:24b-cloud
- 12
- gemma4:31b-cloud
- 12
- glm-5:cloud
- 12
- kimi-k2.5:cloud
- 12
- minimax-m2.5:cloud
- 12
- nemotron-3-nano:30b-cloud
- 11
- rnj-1:8b-cloud
- 11
9 entries · ordered by Exact as observed in this run · not a ranking
What this taught us about function calling
Most models pass
Six of nine are perfect
For the standard 10-function schema, modern Ollama-cloud models are nearly all correctly compliant. The function-calling protocol has stabilized enough that the model's choice of tool is rarely the bottleneck.
Nemotron's false positives
5 of 10 should-not-call prompts triggered a tool
nemotron-3-nano:30b-cloud calls a tool on half the prompts that explicitly should not call any tool. This is the highest-cost failure mode in production — extra latency, extra API calls, sometimes destructive side effects.
Minimax's wrong tool
One wrong-tool error
minimax-m2.5:cloud is the only model that called a different tool than expected. This appears in tool-routing edge cases rather than core functions.
Headline
Should-call accuracy > should-not-call
Across the panel, tool-not-needed prompts produced the main error concentration. Production validation should include explicit no-tool probes alongside tool-correctness probes.