AI Model Reviewer

DeepSeek V4.1 Flash

DeepSeek. Run in Cline CLI, OpenCode, Raw API call, dsh.

248 runs on 208 prompts · 88% finished

Results

The best run of each of its 208 tests first, then every other run. Filter them in the explorer

219 finished runs

Numbers: finish rate, cost, wall time, draft scores
Runs
248
on 208 prompts
Finished
88%
219 of 248
Median cost, finished run
$0.12
n=61
Median wall time, finished run
1m 16s
n=219
Recorded spend
$8.86
$0.76 on unfinished runs; 180 without a recorded cost

From the numbers

Set against the other models with at least 3 runs (top or bottom third), and its draft-score ranks within single prompts. Nothing here is averaged across prompts.

Strengths

  • 1m 16s (n=219): 4th of 23 models with at least 3 (median wall time of a finished run)
  • Draft score 9.5/10 on Isometric SVG town with sprite sheet (in dsh): ranked 2nd of 11

Weaknesses

  • Draft score 7/10 on MacBook Pro 16" realistic 3D render (Playcode benchmark) (in dsh): ranked 10th of 12
  • Draft score 6.5/10 on MacBook Pro 16" realistic 3D render (Playcode benchmark) (in OpenCode): ranked 11th of 12
  • Draft score 6.5/10 on Lighthouse on a headland at night (in OpenCode): ranked 11th of 12
  • Draft score 7/10 on Pelican riding a bicycle (control) (in dsh): ranked 12th of 13
  • Draft score 6.5/10 on Pelican riding a bicycle (control) (in OpenCode): ranked 13th of 13
  • Draft score 8/10 on Animated pelican riding a bicycle (SMIL/CSS) (in OpenCode): ranked 12th of 12
  • 9 runs ended as the model's own failure (DNF, class MODEL)
  • Other unfinished runs, not counted against the model: 8 HARNESS, 9 INFRA, 3 TIMEOUT

Best and hardest run

Best run

Picked as its best draft-score rank on a prompt.

Hardest run

Picked as its costliest run that did not finish.

Draft scores by prompt

Isometric SVG town with sprite sheetdsh9.5/10 draft2nd of 11
Isometric SVG town with sprite sheetOpenCode9/10 draft8th of 11
Lighthouse on a headland at nightdsh8/10 draft6th of 12
Lighthouse on a headland at nightOpenCode6.5/10 draft11th of 12
MacBook Pro 16" realistic 3D render (Playcode benchmark)dsh7/10 draft10th of 12
MacBook Pro 16" realistic 3D render (Playcode benchmark)OpenCode6.5/10 draft11th of 12
Pelican riding a bicycle (control)dsh7/10 draft12th of 13
Pelican riding a bicycle (control)OpenCode6.5/10 draft13th of 13
Animated pelican riding a bicycle (SMIL/CSS)dsh9/10 draft6th of 12
Animated pelican riding a bicycle (SMIL/CSS)OpenCode8/10 draft12th of 12

By harness

Raw API call16615795%$0.035 n=11m 02s n=157
OpenCode474289%$0.13 n=4219m 07s n=42
dsh221882%$0.14 n=1816m 40s n=18
Cline CLI13215%not recorded: no finished run with a recorded cost14m 35s n=2
All 208 tests and experiment collections

Experiment collections: HTML Files (46) · SVG (173) · Three.js (20) · Blender (5) · Harness lane (248)