> Asking questions that are absurd is like interviewing developers and asking absurd questions on the grounds that it tests creative and critical thinking.

Yes; what's wrong with that?

Do you suppose that it doesn't test those qualities?

The whole point of "AI" is arbitrary task completion. Why isn't a SVG drawing relevant for that?

> The pelican prompt is ridiculous

Yes, deliberately so.

It was never intended as a meaningful benchmark. The surprising thing was that for the first ~12 months performance on the stupid pelican benchmark did seem to correspond to the performance of the models on other tasks.

That pattern no longer holds - Fable 5 and GPT-5.6 have both been out-pelicaned by lesser models now.

As I understand it; the point is to ask for an SVG which would demonstrate a conceptual understanding of what is being asked for and that is an important test IMO.

What sufficiently hard, but useful, problem would you ask the model for?

> Test the LLLM against things you want it to do

I agree, it is ridiculous to ask an LLM to replace an artist.