Finding
The prompt, v1, frozen
Draw a café floor tiled with a Penrose rhombus (P3) tiling, as an SVG.
Whenever a task passes every check in three runs for two different models, it comes off the board. So all of these tasks below are now retired. Including a new task I just introduced: the Penrose cafe floor. I judged the Penrose floors by eye, and a script checks the other three.
| Task | First model | Second model |
|---|---|---|
| Penrose floor | Pareto 26.10 Preview, 3 of 3 | GPT-6.1 Sol, 3 of 3 |
| Café order | DeepSeek 4.1 Flash, 3 of 3 | Pareto 26.10 Preview, 3 of 3 |
| Fluorocaffeine | DeepSeek 4.1 Flash, 3 of 3 | Sonnet 5.5, 3 of 3 |
| Triangle wave | DeepSeek 4.1 Flash, 3 of 3 | GPT-6.1 Sol, 3 of 3 |
Verdict
Two models now pass every check in all three runs on each of these four tasks, so I'm taking them off the board to make room for new ones.
n = 24 runs, two models a task. No test.
The 24 runs cost $1.55. A provider timeout used up one more Penrose run, which cost $0.25 and was bought again.