upvote
I don't see how something being lazy for a decade makes it any less lazy. And lazy still seems right to me: They make a misleading point by omitting to collect and present important data. If the headline read "LLMs disagree on 67%, humans disagree on 75%" it would clearly project something very different.

Granted, there certainly are other unflattering adjectives one could have chosen to describe this instead.

reply
Quick note on the second effect - how LLMs reduce that to a four-category judgment: On 21% of the claims at least two models provide polar-opposite verdicts (at least one model False, and at least one model True). This might be a better measurement of the strict disagreement than the 67% disagreement on the four-bucket rubric.
reply