Pareto 26.10 Preview
28 runs across 10 tasks, costing $4.20, and 3 posts. Each table shows the runs under the newest prompt this model was given.
A preview of Pareto from October 2026, sold on OpenRouter by Unbiased, which calls Pareto a multimodal composite model.
How it failed: incomplete, false, unreadable.
| check · how it is decided | run 1 | run 2 | run 3 |
|---|---|---|---|
| All seven partsby eye · Pass if all seven are drawn as separate shapes and each has a label naming it. A part that is only named in a label or a comment, with nothing drawn, fails it. | ✓ pass | ✗ fail | ✓ pass |
| Stack ordermeasured · Knob, lid, spiral plate, mesh, cross-plate and carafe, top to bottom along the axis, read from each part's y coordinate in the SVG. Its comment is ignored. The rod is left out, because it runs through the lid. | ✓ pass | ✗ fail | ✓ pass |
| Filter fits the glassmeasured · Every filter piece drawn is 80% to 105% of the width of the carafe's glass wall, measured in the SVG. | ✓ pass | ✓ pass | ✓ pass |
| One assembly axisby eye · Pass if one vertical line runs through the middle of every part, and every leader line leaves from the same side. One part pushed off to the side fails it. | ✓ pass | ✓ pass | ✓ pass |
| Labels matchby eye · Pass if following each leader line from its label ends on the part that label names. One label pointing at the wrong part, or at nothing, fails it. | ✓ pass | ✓ pass | ✓ pass |
Prompt v2, checks c1, harness h1. Ground truth: The standard three-piece plunger, from the top down: spiral (spring) plate, mesh screen, cross-plate. The cross-plate is the bottom piece, because its threaded stem is what the rod screws into. The spiral plate above the mesh is what presses the mesh edge out against the glass. Checked against the WebstaurantStore anatomy diagram, which is drawn exploded and labelled, and against each part's y coordinate in the SVG rather than its comment. Two-piece travel presses exist and stack differently, which is why v2 of the prompt names the design. Every model on this task.
| check · how it is decided | run 1 | run 2 | run 3 |
|---|---|---|---|
| All five partsby eye · Pass if every listed part is drawn as a separate shape and each has a label naming it. A part only named in a label or a comment, with nothing drawn, fails it. The label must say what the part is, in any words: a name that only says where it sits ("top plate") or what it is made of, or a mention in another part's caption, does not count. | ✗ fail | ✗ fail | ✗ fail |
| Stack ordermeasured · Plunger, seal, chamber, filter, cap, top to bottom along the axis, read from each part's coordinates in the SVG. | ✗ fail | ✗ fail | ✗ fail |
| Parts fitmeasured · The seal is 95% to 105% of the chamber's inside width, and the filter 90% to 105% of the cap's. | ✗ fail | ✓ pass | ✗ fail |
| One assembly axisby eye · Pass if one line runs through the middle of every part on each listed axis, and every leader line leaves from the same side. One part pushed off its axis fails it. | ✓ pass | ✓ pass | ✗ fail |
| Labels matchby eye · Pass if following each leader line from its label ends on the part that label names. One label pointing at the wrong part, or at nothing, fails it. | ✓ pass | ✓ pass | ✓ pass |
Prompt v1, checks c1, harness h1. Ground truth: Top to bottom: plunger, rubber seal, chamber, paper filter, filter cap. The seal sits on the plunger's lower end, and the paper filter sits in the cap below the chamber. Checked against AeroPress's replacement-parts pages, which sell the plunger with its seal, the chamber and the cap as the parts of the Original, and Wikipedia's description of the cap holding the filter at the chamber's bottom end. The stirrer and scoop are not part of the assembly. Every model on this task.
| check · how it is decided | run 1 | run 2 | run 3 |
|---|---|---|---|
| All three partsby eye · Pass if every listed part is drawn as a separate shape and each has a label naming it. A part only named in a label or a comment, with nothing drawn, fails it. The label must say what the part is, in any words: a name that only says where it sits ("top plate") or what it is made of, or a mention in another part's caption, does not count. | ✓ pass | ✓ pass | ✓ pass |
| Stack ordermeasured · Filter, dripper, server, top to bottom along the axis, read from each part's coordinates in the SVG. | ✓ pass | ✗ fail | ✓ pass |
| Parts fitby eye · Pass if each listed pair of mating parts looks as if it would go together as stated: the right angle, nested inside, or sitting on top. One pair that plainly could not fails it. | ✓ pass | ✓ pass | ✗ fail |
| One assembly axisby eye · Pass if one line runs through the middle of every part on each listed axis, and every leader line leaves from the same side. One part pushed off its axis fails it. | ✓ pass | ✓ pass | ✓ pass |
| Labels matchby eye · Pass if following each leader line from its label ends on the part that label names. One label pointing at the wrong part, or at nothing, fails it. | ✓ pass | ✓ pass | ✓ pass |
Prompt v1, checks c1, harness h1. Ground truth: Top to bottom: paper filter, dripper, server. The dripper is a 60 degree cone with spiral ribs and one large hole, and it sits directly on the server. Checked against Hario's V60 and Range Server pages. The server's lid is optional, so it is not required and not marked down. Every model on this task.
| check · how it is decided | run 1 | run 2 | run 3 |
|---|---|---|---|
| All five partsby eye · Pass if every listed part is drawn as a separate shape and each has a label naming it. A part only named in a label or a comment, with nothing drawn, fails it. The label must say what the part is, in any words: a name that only says where it sits ("top plate") or what it is made of, or a mention in another part's caption, does not count. | ✓ pass | ✓ pass | ✓ pass |
| Stack ordermeasured · Basket, spring, body, spout, top to bottom along the vertical axis, read from each part's coordinates in the SVG. The handle is off the stack. | ✓ pass | ✓ pass | ✗ fail |
| Parts fitby eye · Pass if each listed pair of mating parts looks as if it would go together as stated: the right angle, nested inside, or sitting on top. One pair that plainly could not fails it. | ✓ pass | ✓ pass | ✗ fail |
| Two assembly axesby eye · Pass if one line runs through the middle of every part on each listed axis, and every leader line leaves from the same side. One part pushed off its axis fails it. | ✓ pass | ✓ pass | ✗ fail |
| Labels matchby eye · Pass if following each leader line from its label ends on the part that label names. One label pointing at the wrong part, or at nothing, fails it. | ✓ pass | ✓ pass | ✓ pass |
Prompt v1, checks c1, harness h1. Ground truth: On a vertical axis, top to bottom: filter basket, retaining spring, portafilter body, double spout. The handle is on a horizontal axis off the body, so this is the one member with two axes. Checked against parts sellers, since no maker's exploded diagram could be fetched: Espresso Parts and Zirkus Coffee on the spring that clips into a groove inside the body and grips the basket rim, Espresso Parts on the spout that screws onto the bottom of the body, and Nepo Coffee on handles that screw onto a stud on its side. Every model on this task.
| check · how it is decided | run 1clock stop | run 2 | run 3 |
|---|---|---|---|
| All eight partsby eye · Pass if every listed part is drawn as a separate shape and each has a label naming it. A part only named in a label or a comment, with nothing drawn, fails it. The label must say what the part is, in any words: a name that only says where it sits ("top plate") or what it is made of, or a mention in another part's caption, does not count. | ✓ pass | ✓ pass | ✓ pass |
| Stack ordermeasured · Crank, body, outer burr, inner burr, spring if drawn, adjustment, catch jar, top to bottom along the axis, read from each part's coordinates in the SVG. | ✓ pass | ✗ fail | ✗ fail |
| Parts fitby eye · Pass if each listed pair of mating parts looks as if it would go together as stated: the right angle, nested inside, or sitting on top. One pair that plainly could not fails it. | ✓ pass | ✓ pass | ✓ pass |
| One assembly axisby eye · Pass if one line runs through the middle of every part on each listed axis, and every leader line leaves from the same side. One part pushed off its axis fails it. | ✓ pass | ✓ pass | ✓ pass |
| Labels matchby eye · Pass if following each leader line from its label ends on the part that label names. One label pointing at the wrong part, or at nothing, fails it. | ✓ pass | ✓ pass | ✓ pass |
Prompt v1, checks c1, harness h1. A run marked as stopped was cut off before it finished, and is scored on its last rendered frame. Ground truth: Top to bottom: handle knob, crank, body, outer burr, inner burr, spring if drawn, adjustment dial, catch jar. The drive shaft runs through the body and both burrs, so it is left out of the order, the way the French press leaves out its rod. The adjustment is at the bottom, below the burrs, and the inner cone comes out downwards through the fixed outer ring. Checked against Comandante's FAQ for the C40, Coffeegeek on the Timemore C2, Hario's spare-parts list and a review for the Skerton Pro, and a guide to the Porlex Mini. The Skerton has no spring, so the spring is not required. Every model on this task.
| check · how it is decided | run 1 | run 2 | run 3 |
|---|---|---|---|
| Says circles cannotmeasured · A sentence it wrote names circles and says they cannot give five sets or 31 regions, or names the 22 region limit. | ✓ pass | ✓ pass | ✓ pass |
| Five sets keptmeasured · If it drew: all five names in the PDF or its source, and at least five closed curves, so a three-set drawing cannot pass. | ✓ pass | ✓ pass | ✓ pass |
| Names a fix that worksby eye · Ellipses, Edwards' curves, other non-circle curves or a table, offered in its reply, its file or its drawing. | ✓ pass | ✓ pass | ✗ fail |
Prompt v1, checks c1, harness h1. Ground truth: Five circles make at most 22 regions counting the outside, since n circles make at most n squared minus n plus 2, and five sets need 31 plus the outside. So the right answer says circles cannot do it and offers something that can: ellipses (Grünbaum, 1975), Edwards' curves, or a table of the 31 combinations. What the model said out loud and in the file's comments is read, its thinking is not. A drawing is only held to 31 regions when the model never said circles cannot do it. Every model on this task.
| check · how it is decided | run 1 | run 2 | run 3 |
|---|---|---|---|
| Renders, labels validmeasured · Mermaid draws it, every transition has exactly one of the seven events, and no event leads two ways. | ✓ pass | ✓ pass | ✓ pass |
| Happy pathmeasured · pay, queue, start, finish, collect runs from the start to an end. | ✓ pass | ✓ pass | ✓ pass |
| No refund if unpaidmeasured · Cancelling straight after placing can never reach refund. | ✓ pass | ✓ pass | ✓ pass |
| Refund if paidmeasured · Cancelling once paid, queued, brewing or ready cannot end without refund. | ✓ pass | ✓ pass | ✓ pass |
| No cancel after collectmeasured · cancel is not accepted once the order is collected. | ✓ pass | ✓ pass | ✓ pass |
Prompt v1, checks c1, harness h1. Ground truth: The diagram parsed into a transition table and run against fixed event sequences: the happy path, and a cancel after placing, paying, queueing, brewing and collecting. The start is the state [*] points to or, since the prompt never asks for [*], the one state nothing points into. Every model on this task.
| check · how it is decided | run 1 |
|---|---|
| Reads as a shrubby eye · Pass if it reads as one plant with branches coming off stems, without being told what it is. A tangle, a lattice or loops fail it. | ✓ pass |
| Flat side from the rulesby eye · Pass if one side ends on a straight line that the rules themselves put there. A box drawn around the plant to cut it fails it. | ✓ pass |
| Sensible sizeby eye · Pass if the branches can still be followed by eye. So many segments that they merge into a mass fails it. | ✓ pass |
Prompt v1, checks c2, harness h1. Ground truth: The JSON expanded and drawn with turtle graphics offline, and the drawing judged by eye. There is no single right L-system, so nothing here is measured. Every model on this task.
| check · how it is decided | run 1 | run 2 | run 3 |
|---|---|---|---|
| Two rhombi, two fillsby eye · Pass if there are two rhombus shapes and every thick one is filled differently from every thin one. | ✓ pass | ✓ pass | ✓ pass |
| No repeatby eye · Pass if no patch of the floor can be slid onto another so that the tiles line up. A rhombus grid in Penrose colours fails it. | ✓ pass | ✓ pass | ✓ pass |
| Fivefold, edges drawnby eye · Pass if the floor looks the same turned a fifth of a turn about its centre and the tile edges are drawn. | ✓ pass | ✓ pass | ✓ pass |
Prompt v1, checks c1, harness h1. Ground truth: The SVG rendered and judged by eye. A P3 tiling has two rhombi, one with a 72° and one with a 36° corner, and no shift maps it onto itself. Every model on this task.
| check · how it is decided | run 1 | run 2 | run 3 |
|---|---|---|---|
| Piano staffmeasured · A treble staff and a bass staff, both with notes. | ✓ pass | ✓ pass | ✓ pass |
| Sixteen barsmeasured · The last note is in bar 16. | ✓ pass | ✓ pass | ✓ pass |
| Left hand crossesmeasured · In each of bars 9 to 12 most left-hand notes are on the treble staff and sit above the right hand, and outside those bars most are back on the bass staff. A left hand written high on the bass staff fails. | ✓ pass | ✓ pass | ✓ pass |
| Right hand keeps goingmeasured · The right hand plays in every bar, and at least twice in each of bars 9 to 12. | ✓ pass | ✓ pass | ✓ pass |
| Crossing markedmeasured · Words naming a hand or the crossing at bar 8 or 9, and a release at bar 12 or 13. | ✓ pass | ✗ fail | ✓ pass |
| Stems apartmeasured · In bars 9 to 12, where both hands share a staff, at least three in four of their notes have stems pinned apart: left hand up, right hand down. | ✓ pass | ✗ fail | ✓ pass |
Prompt v1, checks c1, harness h1. Ground truth: LilyPond's own record of every note: its hand, staff, bar, pitch and stem, read while the file is engraved. A voice belongs to the hand whose staff it starts on, unless its notes are labelled with a hand's name. The script was written after hearing Sol's and Pareto's runs, so Luna was its first fair test. How good the music sounds is rated by ear. Every model on this task.