Thanks for sharing another solid data point. I fear you won't get an answer from my experience [0]. Unfortunately, the blog post decided to forgo the very models that I found to be the worst offenders:
reply> Here is what Qwen3.6-35B-A3B via Openrouter provided for a sloth riding a skateboard: https://imgur.com/a/Dy8fvR5
> Like Grok 4 Fasts attempt at a mushroom in a rowboat, it is barely recognisable as anything despite both Qwen3.6-35B-A3B and Grok 4 Fast having no issue with more popular (i.e. benchmarked) examples. [...]
> And here is Opus 4.7 [which simonw claimed to provide a worse pelican vs Qwen], again via Openrouter: https://imgur.com/a/Qus1Enf
Anyone who hasn't witnessed such deltas either hasn't looked at enough examples, a sufficient variety of models, or both. And they are, unfortunately, not limited to "SVGMaxxing", but a wide range of evals.
Does asking for a dagger help?
replyIf you look at the raster image ChatGPT generated, that is fine. It is just this example (and other simple SVG icons I have asked for) result in pretty bad SVGs. It just makes me highly suspicious that the LLMs are learning shape primitives and extrapolating to new shapes, vs just having a big dictionary of prior examples and stitching them together.
reply