AI Model Reviewer

GPT-6 Astra

OpenAI. Run in Codex CLI, OpenCode, Raw API call.

264 runs on 215 prompts · 98% finished

Results

The best run of each of its 215 tests first, then every other run. Filter them in the explorer

260 finished runs

Numbers: finish rate, cost, wall time, draft scores
Runs
264
on 215 prompts
Finished
98%
260 of 264
Median cost, finished run
$0.87
n=258
Median wall time, finished run
5m 14s
n=260
Recorded spend
$743.43
$1.26 on unfinished runs; 3 without a recorded cost

From the numbers

Set against the other models with at least 3 runs (top or bottom third), and its draft-score ranks within single prompts. Nothing here is averaged across prompts.

Strengths

  • 260 of 264 runs finished: 5th of 24 models with at least 3 (finish rate)
  • Draft score 9.5/10 on Pelican riding a bicycle (control) (in Codex CLI): ranked 1st of 13
  • Draft score 9/10 on MacBook Pro 16" realistic 3D render (Playcode benchmark) (in OpenCode): ranked 2nd of 12
  • Draft score 9.5/10 on Animated pelican riding a bicycle (SMIL/CSS) (in OpenCode): ranked 2nd of 12
  • Draft score 9.5/10 on Isometric SVG town with sprite sheet (in Codex CLI): ranked 2nd of 11
  • Draft score 9.5/10 on Isometric SVG town with sprite sheet (in OpenCode): ranked 2nd of 11

Weaknesses

  • 3 runs ended as the model's own failure (DNF, class MODEL)
  • Other unfinished runs, not counted against the model: 1 INFRA

Best and hardest run

Best run

Picked as its best draft-score rank on a prompt.

Hardest run

Picked as its costliest run that did not finish.

Draft scores by prompt

Isometric SVG town with sprite sheetCodex CLI9.5/10 draft2nd of 11
Isometric SVG town with sprite sheetOpenCode9.5/10 draft2nd of 11
Lighthouse on a headland at nightCodex CLI8/10 draft6th of 12
Lighthouse on a headland at nightOpenCode8/10 draft6th of 12
MacBook Pro 16" realistic 3D render (Playcode benchmark)OpenCode9/10 draft2nd of 12
MacBook Pro 16" realistic 3D render (Playcode benchmark)Codex CLI8.5/10 draft6th of 12
Pelican riding a bicycle (control)Codex CLI9.5/10 draft1st of 13
Pelican riding a bicycle (control)OpenCode9/10 draft6th of 13
Animated pelican riding a bicycle (SMIL/CSS)OpenCode9.5/10 draft2nd of 12
Animated pelican riding a bicycle (SMIL/CSS)Codex CLI9/10 draft6th of 12

By harness

Codex CLI6767100%$5.99 n=6725m 22s n=67
OpenCode504998%$4.30 n=4918m 28s n=49
Raw API call14714498%$0.61 n=1423m 46s n=144
All 215 tests and experiment collections

Experiment collections: HTML Files (63) · SVG (138) · Three.js (41) · Blender (11) · Harness lane (264)