Codex vs Claude — native audit of a 425K LOC project · V1

Agent and CLI Testing

Seven OpenAI Codex models, three Anthropic Claude models, and one OpenCode wrapper run the same eight-category code audit against the production platform project (3,877 Python files, 557,386 LOC, 21,978 test functions). Each model gets the same 900-second budget and produces a structured audit report.

Opus unique findings: 12
Claude Code completion: 100%
Codex CLI completion: 43%
OpenCode completion: 0%

One prompt, three vendor channels

Each wrapper was given the same eight-category audit: Project Structure, Code Quality, Security, Testing, Documentation, Dependencies, Performance, DevOps. The model was free to use any tools it had available (shell, read, grep, sub-agent spawning, MCP servers). The output was scored on completion, depth of unique findings, file:line citation density, and overall actionability.

Cross-vendor comparison is structurally impossible. Codex CLI has no Anthropic provider. Claude Code has no OpenAI provider. OpenCode required an Anthropic API key it had no working route to (the supplied 108-char key was expired; OpenCode does not fall back to OAuth). Each wrapper is a sealed family channel — the wrapper itself enforces the family boundary.

The full results table

Unique findings

claude-opus-4-6: 12
gpt-5.1-codex-mini: 6
claude-sonnet-4-5: 6
claude-haiku-4-5: 5
gpt-5.2-codex: 4
gpt-5.1-codex-max: 4
gpt-5.3-codex: 0
gpt-5.3-codex-spark: 0
gpt-5.2: 0
o3: 0
OC-claude-haiku: 0
OC-claude-sonnet: 0
OC-claude-opus: 0

13 entries · ordered by Unique findings as observed in this run · not a ranking

What the audit showed

U-shaped quality curve

Mini and Opus observed higher — mid-large lower

The smallest Codex (mini, 9/10) recorded higher scores than the mid/large Codex variants (gpt-5.3-codex 4/10). Larger reasoning models over-explored and ran out of context before synthesizing. The bottom of the curve is the middle, not the edges.

Sub-agent orchestration

Only Opus deployed sub-agents

Despite Codex CLI supporting collab=true, no Codex model used sub-agents. Only claude-opus-4-6 spawned 6 parallel investigators (structure, security, testing, dependencies, code quality, documentation), producing the 12 unique findings no other model surfaced.

Haiku's output strategy

Short summary + long artifact = 22× efficiency

Haiku used a "short inline summary + long file artifact" pattern, scoring identically to Sonnet's 793-line full-inline output at 22× the context efficiency. Output strategy matters as much as model size.

Cost-quality combo

Opus + Mini covers ~95%

Either alone covers ~70–80% of findings. Combined they catch ~95%. Mini is the cost-efficient half; Opus is the depth half. Neither alone is sufficient for production-grade audits.

The 12 unique Opus findings

Numbers the other 12 model runs never produced. This is what "depth of investigation" looks like in measurable terms.

0.48% root coverage / 1.90% module coverage from coverage.xml despite 21,978 tests
57 bare except: clauses across 30 files (also caught by Haiku)
120+ orphaned audit report files; 80MB+ coverage artifacts tracked in git
7.8 MB logging binary accidentally committed at repo root
Pre-commit coverage hooks set to stages: [manual] — never run automatically
--cov-fail-under=0 effectively disables coverage enforcement
OpenTelemetry 0.60b1 (beta) in production; ChromaDB pinned 2 major versions behind (1.3.5)
Frontend Node 20 prod vs Node 24 CI mismatch
eval/exec usage in the observability worker
Case-collision risk between audit_report.md and Audit_Report.md
SQL injection via f-string at the query layer, 601
Hardcoded production IPs at sigmatrace/config/settings.py:98, 108, 118, 129, 157