Gallery, prompt v1

Espresso Rag, with the left hand crossing over

One frozen prompt and every model that has run it, newest first.

The prompt, v1, frozen

Write, in LilyPond, 16 bars of café piano music called "Espresso Rag" in which the left hand crosses over the right hand for bars 9 to 12.

Every model, its first three runs.

One square a check, in this order:

  1. Piano staff
  2. Sixteen bars
  3. Left hand crosses
  4. Right hand keeps going
  5. Crossing marked
  6. Stems apart

passed, failed, undecided. A round mark is a check I judged by eye, and a diamond one a check a model judged.

Claude Haiku 5.5
Pareto 26.10 Preview
GPT-6 Luna
GPT-6.1 Sol

What a run cost, against checks passed.

One dot a model, over its three runs above. Further right costs more. The gap from $0.01 to $0.1 is as wide as the gap from $0.1 to $1, so the cheap models are not squeezed into a corner.

0%25%50%75%100%$0.01$0.03$0.1cost per runGPT-6.1 Sol: $0.038 a run, 18 of 18 checks passedGPT-6.1 SolPareto 26.10 Preview: $0.14 a run, 16 of 18 checks passedPareto 26.10 PreviewClaude Haiku 5.5: $0.011 a run, 14 of 18 checks passedClaude Haiku 5.5GPT-6 Luna: $0.0055 a run, 12 of 18 checks passedGPT-6 Luna
Every dot answered this prompt with these checks, so they compare. A run that produced nothing still cost what it cost, and passed none of the checks.

How it is checked

LilyPond's own record of every note: its hand, staff, bar, pitch and stem, read while the file is engraved. A voice belongs to the hand whose staff it starts on, unless its notes are labelled with a hand's name. The script was written after hearing Sol's and Pareto's runs, so Luna was its first fair test. How good the music sounds is rated by ear.

Piano staffmeasuredA treble staff and a bass staff, both with notes.
Sixteen barsmeasuredThe last note is in bar 16.
Left hand crossesmeasuredIn each of bars 9 to 12 most left-hand notes are on the treble staff and sit above the right hand, and outside those bars most are back on the bass staff. A left hand written high on the bass staff fails.
Right hand keeps goingmeasuredThe right hand plays in every bar, and at least twice in each of bars 9 to 12.
Crossing markedmeasuredWords naming a hand or the crossing at bar 8 or 9, and a release at bar 12 or 13.
Stems apartmeasuredIn bars 9 to 12, where both hands share a staff, at least three in four of their notes have stems pinned apart: left hand up, right hand down.

When this saturates: Hand it an existing rag and ask for the same music with the hands swapped in bars 9 to 12, so every note has to stay where it is.

Failures seen here: incomplete (fails when the model leaves out something the brief asked for); unreadable (fails when the content may be right, but a reader can't make it out); false (fails when the model gets a fact wrong, such as a part in the wrong place or at the wrong size). The same kinds across every task.

The rules behind all of thisThis task's prompt, checks and any checker, on GitHub