Why? Don’t you think the steady improvement in pelican bicycling SVG art is itself a worthy field of endeavor for LLMs to be optimized for?
replyYes, Simon has made pelicans a benchmark and now models have grown very good at rendering them. Is there a reason to believe that this isn’t manifesting as useful skill in general in SVG generation?
The sample is too small. As long as the models are reasonable close enough to each other there is really nothing you can learn from this benchmark when its just one prompt.
replyIt would perhaps be slightly interesting if the prompt was run 100 times for each model. I doubt the pelicans would look identical.