How good are models at separating music from background coffee machine noise?

Finding

I played an espresso machine over piano music (a previous sample generated by a model) and asked five models to take it out. They can't listen, so they have to measure. As I have the music sample and the esspresso machine isolated, I used them for the checker to check how good a job the models did.

The prompt, v1, frozen

I run a café and I recorded the background music we play, but the espresso machine was going the whole time and you can hear it over the music. The recording is cafe.wav. Take the coffee machine out and leave the music as it is. Save the result as clean.wav, at the same sample rate and length.

	Hum down	Rush down	Music kept	Sounds right to me
GPT-6.1 Terra	1 of 3	1 of 3	3 of 3	0 of 3
Opus 5.5	2 of 2	0 of 2	2 of 2	0 of 2
Gemini 4 Argon	1 of 3	1 of 3	2 of 3	0 of 3
GPT-6.1 Sol	0 of 2	2 of 2	0 of 2	0 of 2
Kimi K3.1 Pro	0 of 3	0 of 3	0 of 3	0 of 3

The coffee machine taken out of the café's music, by gpt-6.1-terra — GPT-6.1 Terra, run 1, the only run that passed all four measured checks. I gave it 2 out of 5.$0.08

The coffee machine taken out of the café's music, by claude-opus-5.5 — Opus 5.5, run 1. The hum is down and the music is kept, but the rush is still there. I gave it 3 out of 5, the best I gave.$0.08

The coffee machine taken out of the café's music, by gemini-4-argon — Gemini 4 Argon, run 2. The hum and the rush are down, but it took a lot of the piano with them. I gave it 2 out of 5.$0.64

The coffee machine taken out of the café's music, by gpt-6.1-sol — GPT-6.1 Sol, run 1. The rush is down, but the hum is still there and the piano lost too much. I gave it 2 out of 5.$0.06

The coffee machine taken out of the café's music, by kimi-k3.1-pro — Kimi K3.1 Pro, run 1. It passed none of the measured checks. I gave it 1 out of 5, the worst I gave.$0.22

I gueessed not model would pass and that held. All models really struggle with that tasks, some do better than others.

The 13 runs cost $2.82.