For posterity, the memetic ugly food can be controlled and mostly filtered out with prompt engineering. Here's a quick ad hoc test example which (hopefully) uses the new model: https://chatgpt.com/share/6aebf030-9d38-83e8-a103-3509883905...
First image is fine, but attempts to make it more wacky turn it to the AI food slop. So then followup prompts deslopify it, mostly successfully.
I think the way they'll solve it in the end is to create an ultra-high-fidelity simulation of the world including modelling taste and other senses on the level of physics, and then incorporate all of this into the training rewards.
I think this is how the machines know what tasty wheat tasted like.