31 cloud models × 3 wrappers on the same audit · V1

Cloud Wrapper Comparison

Earliest Sigma5C wrapper comparison observation, predating the formal Wrapper Comparison Observatory. The same the production platform 8-category code audit run by every Ollama-cloud model through each of three agentic wrappers — Claude Code, Codex CLI, OpenCode — with a 15-minute timeout per cell. 93 runs total. Surfaces three failure modes: HTTP 429 rate limits, tool-call format mismatch, and preflight timeouts.

OpenCode pass: 45%
Claude Code pass: 39%
Codex CLI pass: 6%
model passed all 3: 1

Full quality ranking

Models 1–17 produced at least one passing run; models 18–31 produced none. The "Reports" column shows wrapper count (out of 3) that successfully completed the audit.

Avg Quality

qwen3-coder:480b-cloud: 8.5
deepseek-v3.1:671b-cloud: 8.5
devstral-2:123b-cloud: 8.5
kimi-k2-thinking:cloud: 8.5
glm-4.7:cloud: 8.5
minimax-m2.1:cloud: 8
kimi-k2.5:cloud: 8
minimax-m2.5:cloud: 7.7
gpt-oss:120b-cloud: 7.5
glm-4.6:cloud: 7.5
gpt-oss:20b-cloud: 7.5
ministral-3:8b-cloud: 7
gemini-3-flash-preview:cloud: 7
ministral-3:3b-cloud: 6.5
mistral-large-3:675b-cloud: 6.5
gemma3:4b-cloud: 6
minimax-m2:cloud: 6

17 entries · ordered by Avg Quality as observed in this run · not a ranking

What the data revealed

Failure 1 — 429 storm

Codex CLI loses to rate limits

Codex was destroyed by HTTP 429 in 29 of 31 runs. The wrapper does not back off aggressively enough for Ollama-cloud burst limits. Recovery requires explicit retry-after handling that Codex CLI does not implement.

Failure 2 — tool-call format

5 models emit XML, not JSON

cogito-2.1, deepseek-v3.2, gemma3:4b/12b/27b emit XML or text tool-call syntax instead of OpenAI-compatible JSON. The wrappers cannot parse this. The fix is at the model side — these families need OpenAI-shape function-call training updates.

Failure 3 — preflight

4 models hit assorted issues

glm-4.6, glm-5, qwen3-coder-next, qwen3-vl:235b — each fails at a different point (preflight token-count probes, vision-only restrictions, etc.). These are wrapper-specific architecture issues, not model issues.

The one model that passed all three wrappers was minimax-m2.5. It scored 7.7 overall — not the absolute top, but it is the only model that returned a usable audit through every wrapper. For multi-wrapper production deployments, model selection should prioritize broad wrapper compatibility, not pure quality.

Smaller open-source GPT model wins

gpt-oss:20b-cloud (7.5, 2/3 wrappers) recorded higher reliability than gpt-oss:120b-cloud (7.5, 1/3 wrappers). Same model family, smaller scored higher on the cross-wrapper axis. This matches the U-shaped quality curve also observed in the codex_vs_claude_native study — small and very-large models clear the bar consistently; the middle does not.

31 cloud models × 3 wrappers on the same audit · V1

Full quality ranking

What the data revealed

Codex CLI loses to rate limits

5 models emit XML, not JSON

4 models hit assorted issues

Smaller open-source GPT model wins

Models

Families

Companies

Related observations