Do models know how to stir a coffee?

In short

Some models have the spoon flat on the surface to stir and others have the spoon leaving the cup while it stirs.

Finding

Some models have the spoon flat on the surface to stir and others have the spoon leaving the cup while it stirs.

I asked three models for a cup of coffee with a spoon stirring in it, in HTML and CSS. This is as I want more dynamic behaviour checks on the site.

Sonnet 5.5, run 1. Bowl in the coffee, handle over the near rim.$0.01
Sonnet 5.5, run 2. Upright, less paint.$0.01

Sonnet stirs it in all three runs.

GLM 5.3 FlashX, run 1. Bowl at the back rim, handle across the mouth.$0.04
GLM 5.3 FlashX, run 3. The same layout on a mug.$0.08

GLM stirs, but the spoon leaves the cup.

DeepSeek 4.1 Flash, run 1. The spoon is an outline on the surface.$0.28
DeepSeek 4.1 Flash, run 2. An arm turns about the centre.$0.27

DeepSeek's spoon is lying on the surface.

every model
modelrunsHTML and CSS onlyA CSS loop that moves somethingReads as a cup seen intoThe spoon does the stirringReads as inside the cupHandle stays on the bowl
Claude Sonnet 5.533 of 33 of 33 of 33 of 33 of 33 of 3
GLM 5.3 FlashX33 of 33 of 33 of 33 of 30 of 33 of 3
DeepSeek 4.1 Flash33 of 33 of 33 of 30 of 32 of 31 of 2 · 1 n/d

Each cell is runs that passed that check, of runs where it was decided. Every run, with pictures.

Failed How many Dimension Property Mode
The spoon does the stirring DeepSeek, 3 runs function behaviour missing
Reads as inside the cup GLM, 3 runs validity relation transposed
Reads as inside the cup DeepSeek run 1 validity relation transposed
Handle stays on the bowl DeepSeek run 1 validity geometry distorted

Verdict

DeepSeek's failure is a missing behaviour, the failure of GLM is that the spoon leaves the cup while stirring.

n = 3 runs a model. No test.

The nine runs cost $0.96.

By email

How it failed

broken, false

Older, 18 Oct

What do models like to do when given the opportunity?

Newer, 20 Oct

How good are models at fixing things?