GLM-5.3
Z.ai. Run in Claude Code, OpenCode, Raw API call. Served via Fireworks; GLM has no coding CLI of its own, so its home track runs in Claude Code (Z.ai's recommended harness), labelled as such.
110 runs on 77 prompts · 41% finished
Results
The best run of each of its 77 tests first, then every other run. Filter them in the explorer
45 finished runs
Video printing press collage hyperframes
GLM-5.3
OpenCode 1.18.32
skills off
Failed · Stopped by us
No rendered video or project published
Isometric SVG town with sprite sheet
GLM-5.3
Claude Code 2.1.280
skills off
Failed · Harness and model mismatch
Pelican riding a bicycle (control)
GLM-5.3
Claude Code 2.1.280
skills off
Failed · Harness and model mismatch
Animated pelican riding a bicycle (SMIL/CSS)
GLM-5.3
Claude Code 2.1.280
skills off
Failed · Infrastructure failure
Animated pelican riding a bicycle (SMIL/CSS)
GLM-5.3
Claude Code 2.1.280
skills off
Failed · Harness and model mismatch
MacBook Pro 16" realistic 3D render (Playcode benchmark)
GLM-5.3
Claude Code 2.1.280
skills off
Failed · Harness and model mismatch
Lighthouse on a headland at night
GLM-5.3
Claude Code 2.1.280
skills off
Failed · Harness and model mismatch
BMW M4 Competition, side view (flat vector)
GLM-5.3
Claude Code 2.1.280
skills off
Failed · Harness and model mismatch
California brown pelican riding a bicycle (detailed spec)
GLM-5.3
Claude Code 2.1.280
skills off
Failed · Harness and model mismatch
DualSense controller, max fidelity
GLM-5.3
Claude Code 2.1.280
skills off
Failed · Harness and model mismatch
Numbers: finish rate, cost, wall time, draft scores
- Runs
- 110
- on 77 prompts
- Finished
- 41%
- 45 of 110
- Median cost, finished run
- $1.08
- n=31
- Median wall time, finished run
- 17m 09s
- n=45
- Recorded spend
- $73.00
- $20.54 on unfinished runs; 61 without a recorded cost
From the numbers
Set against the other models with at least 3 runs (top or bottom third), and its draft-score ranks within single prompts. Nothing here is averaged across prompts.
Strengths
- Draft score 9.5/10 on Pelican riding a bicycle (control) (in OpenCode): ranked 1st of 13
- Draft score 9/10 on MacBook Pro 16" realistic 3D render (Playcode benchmark) (in OpenCode): ranked 2nd of 12
- Draft score 9.5/10 on Isometric SVG town with sprite sheet (in OpenCode): ranked 2nd of 11
Weaknesses
- 45 of 110 runs finished: 22nd of 24 models with at least 3 (finish rate)
- 17m 09s (n=45): 20th of 23 models with at least 3 (median wall time of a finished run)
- Draft score 5.5/10 on Lighthouse on a headland at night (in OpenCode): ranked 12th of 12
- 44 runs ended as the model's own failure (DNF, class MODEL)
- Other unfinished runs, not counted against the model: 9 HARNESS, 4 INFRA, 7 TIMEOUT, 1 ERROR
Best and hardest run
Best run
Picked as its best draft-score rank on a prompt.
-
GLM-5.3, OpenCode, Pelican riding a bicycle (control)
OpenCode 1.18.32, max effort, via Fireworks
skills off
9m 06s
$0.33 reported by OpenCode est. $0.33
OK
AI judge 9.5/10 Draft scorerubric
Hardest run
Picked as its costliest run that did not finish.
-
Did not finish · re-run queued
GLM-5.3, OpenCode, video_printing_press_collage_hyperframes
OpenCode 1.18.32, max effort, via Fireworks
skills off
Stopped by us
3h 22m $7.37 reported by OpenCode est. $7.37
Draft scores by prompt
| Isometric SVG town with sprite sheet | OpenCode | 9.5/10 draft | 2nd of 11 |
|---|---|---|---|
| Lighthouse on a headland at night | OpenCode | 5.5/10 draft | 12th of 12 |
| MacBook Pro 16" realistic 3D render (Playcode benchmark) | OpenCode | 9/10 draft | 2nd of 12 |
| Pelican riding a bicycle (control) | OpenCode | 9.5/10 draft | 1st of 13 |
| Animated pelican riding a bicycle (SMIL/CSS) | OpenCode | 9/10 draft | 6th of 12 |
By harness
| OpenCode | 24 | 17 | 71% | $1.08 n=17 | 26m 23s n=17 |
|---|---|---|---|---|---|
| Claude Code | 29 | 14 | 48% | $1.07 n=14 | 21m 34s n=14 |
| Raw API call | 57 | 14 | 25% | not recorded: no finished run with a recorded cost | 11m 22s n=14 |
All 77 tests and experiment collections
- Isometric SVG town with sprite sheet (4 runs)
- Pelican riding a bicycle (control) (3 runs)
- Animated pelican riding a bicycle (SMIL/CSS) (4 runs)
- MacBook Pro 16" realistic 3D render (Playcode benchmark) (3 runs)
- Lighthouse on a headland at night (3 runs)
- Fjord scene (Three.js) (2 runs)
- Video printing press collage remotion (1 run)
- Spongebob (1 run)
- Desert Dunes at Night (1 run)
- Sales Dashboard Card (1 run)
- Calendar Month View (1 run)
- Exploded Burger (1 run)
- Bauhaus Composition (1 run)
- Basketball Slam Dunk (1 run)
- Battery Level Icons (1 run)
- Fruit Sales Bar Chart (1 run)
- Bohr Model of an Atom (1 run)
- Flat App Icon Set (1 run)
- Alien Planet (1 run)
- Acoustic Guitar (1 run)
- Sleeping Curled Cat (2 runs)
- GPU fluid sim (1 run)
- Pelican car france (2 runs)
- Dolphin vlog bicycle (2 runs)
- Whale clouds (2 runs)
- BMW M4 Competition, side view (flat vector) (3 runs)
- California brown pelican riding a bicycle (detailed spec) (3 runs)
- DualSense controller, max fidelity (3 runs)
- Xbox 360 controller (6 runs)
- Slime softbody (1 run)
- Game battleship (1 run)
- Stillwater pond (1 run)
- Procedural armor suit (1 run)
- Creature robot blueprint (1 run)
- Artwork style pass (1 run)
- Mechanical bee (1 run)
- Humanoid walk cycle (1 run)
- Humanoid motion cycle (1 run)
- Video printing press collage hyperframes (1 run)
- Cycling Peloton (1 run)
- Cyberpunk Alley (1 run)
- Corgi Catching a Frisbee (1 run)
- Chalkboard Coffee Lettering (1 run)
- Top-Down City Park (1 run)
- Circuit Board Close-Up (1 run)
- Chat Conversation UI (1 run)
- Chef Flipping a Pan (1 run)
- Chessboard Mid-Game (1 run)
- Chameleon on a Branch (1 run)
- Container Ship (1 run)
- Canyon River (1 run)
- Campus Map (1 run)
- Butterfly Life Cycle (1 run)
- Breakfast Skillet (1 run)
- Bubble Tea (1 run)
- Labelled Bicycle Diagram (1 run)
- Boxing Ring (1 run)
- Baseball Stadium (1 run)
- Barn Owl in Flight (1 run)
- Ballet Dancer (1 run)
- Autumn Forest Path (1 run)
- Stargazer with Telescope (1 run)
- Floating Astronaut (1 run)
- Art Deco Skyscraper (1 run)
- Ant Colony Cutaway (1 run)
- Illustrated Alphabet A B C (1 run)
- Twin-Bell Alarm Clock (1 run)
- Elephant Calf Mud Bath (2 runs)
- 004 Brick Breaker Neon (1 run)
- 003 Orbital Coffee Landing (2 runs)
- Red Fox in Snow (2 runs)
- 002 Pendulum Wave (2 runs)
- 005 Focus Timer Orbit (1 run)
- 001 Ink Bloom Garden (1 run)
- Giraffe Side Profile (1 run)
- Octopus Juggling Teacups (1 run)
- X16 engine (2 runs)
Experiment collections: HTML Files (20) · SVG (86) · Three.js (13) · Blender (2) · Harness lane (110)