Models as data workers · V1
Model observation
- Task suites
- 4
- Model tags
- 16
Patterns by task
The same model recorded different scores across domain, SQL and workflow tasks. Each suite is shown separately to keep those task conditions visible.
Every model in the recorded vision suite received a zero score. This observation is limited to that task set and its recorded execution conditions.
How to read the scores
The model sets differ by suite. Scores include zero outcomes and do not separate model behavior from parser, execution or task-design effects. Catalog links identify exact name matches; they do not establish a historical checkpoint.
Vision suite
- deepseek-ocr:latest
- 0.000
- glm-ocr:latest
- 0.000
- medgemma1.5:latest
- 0.000
- ministral-3:3b
- 0.000
- qwen3-vl:2b
- 0.000
- qwen3-vl:4b
- 0.000
Each bar averages the recorded overall scores for that exact model in this suite, including zero scores. Values are rounded to three decimal places.
Domain suite
- granite4:3b
- 0.025
- lfm2.5-thinking:latest
- 0.672
- nemotron-3-nano:4b
- 0.684
- olmo-3:7b-instruct
- 0.790
- phi4-mini-reasoning:latest
- 0.785
- qwen3.5:0.8b
- 0.080
- qwen3.5:2b
- 0.000
- qwen3.5:4b
- 0.163
- qwen3.5:latest
- 0.128
- rnj-1:latest
- 0.652
Each bar averages the recorded overall scores for that exact model in this suite, including zero scores. Values are rounded to three decimal places.
SQL suite
- granite4:3b
- 0.200
- lfm2.5-thinking:latest
- 0.051
- nemotron-3-nano:4b
- 0.196
- olmo-3:7b-instruct
- 0.196
- phi4-mini-reasoning:latest
- 0.011
- qwen3-vl:2b
- 0.022
- qwen3.5:0.8b
- 0.000
- qwen3.5:2b
- 0.084
- qwen3.5:4b
- 0.198
- rnj-1:latest
- 0.187
Each bar averages the recorded overall scores for that exact model in this suite, including zero scores. Values are rounded to three decimal places.
Workflow suite
- granite4:3b
- 0.578
- lfm2.5-thinking:latest
- 0.367
- nemotron-3-nano:4b
- 0.720
- olmo-3:7b-instruct
- 0.667
- phi4-mini-reasoning:latest
- 0.420
- qwen3-vl:2b
- 0.278
- qwen3.5:0.8b
- 0.050
- qwen3.5:2b
- 0.147
- qwen3.5:4b
- 0.217
- rnj-1:latest
- 0.402
Each bar averages the recorded overall scores for that exact model in this suite, including zero scores. Values are rounded to three decimal places.
Models
Related observations
- Tool-Calling Compatibility — 31 cloud models · V1
- Cloud Wrapper Comparison — 31 models × 3 wrappers · V1
- Agent memory design reference · V1
- Capability Ladder — 8 models × 240 cells · V1
- Minimum Viable Model — scaffolding modes · V1
- Visual extraction and scoring observation: local MiniCPM-V 4.5 and 4.6 aliases · V3