The quality of the Gemini pelican was such a step change in one iteration, while the other benchmarks remained quite flat, that I think you are right. Although whether they targeted Pelicans in particular or just svg, I can't say.
replydeleted
replyEvery modern image-generation model can generate a pelican on a bicycle trivially. The point of the test is to generate SVG text that represents an image, which is more complicated.
replyYes, there are ways to convert raster images to SVG for use in training data but it's not a good use of anyone's time.
I don't understand this response. Human artists can and do make SVGs.
reply> Every modern image-generation model can generate a pelican on a bicycle trivially.
replyMistral seems to be the exception. Their new model from a few weeks ago is worse then selfhosted gemma.
I wouldn't wish creating a svg pelican on a bicycle on my worst enemy
reply