undefined

points

[-]

Idea for a website/documentary -- have experts respond to a piece of news, or provide commentary. Put a few expert pieces alongside a few LLM outputs, have people guess/work out which is which. Have the same people tell you why.

If on a website, rank the results; present the 'how I worked it out' info for the best spotters (and you could interview them). Keep the answers secret for a few weeks, then reveal them in a way that the game is still playable.

It's repeatable, every few months you could interview new experts (or the old ones again), get new models.

Kinda like the critical thinking version of images of a pelican on a bike.

by mrandish2 hours ago|

parent|

[-]

I love your idea and would enjoy seeing the results of that controlled experiment.

I'm also interested in the broader impact of using LLMs in place of web search for general Q&A when we want 'to know things'. It's pretty clear the way LLMs are being used for knowledge acquisition now is often less accurate while 'feeling' more certain. Even if we set aside explicit hallucinations, I suspect it's still less accurate.

by jmhmd5 hours ago|

prev|

[-]

Hah, I'm neither a bot nor written with any help from an LLM, but I'll take the fact that you can't tell the difference as a compliment :)

by b40d-48b2-979e5 hours ago|

parent|

[-]

It's not meant as a slight against you, but an observation on people blindly trusting Internet commenters in a time where our trust is (or should be?) at an all-time low for such content, so we should check our priors to ensure what we're consuming can be trusted and is verifiably true as LLMs exude confidently incorrect behavior.

by cscheid3 hours ago|

parent|

[-]

I want to point out that you are posting from a 5-months old account (squarely within a time frame where LLM-powered accounts would be created), with an UUID-sounding username, claiming a 10-year-old professional history in the field, and using those credentials to bring up the possibility that a 14-year-old account claiming to be an MD with `md` in the username and _lots_ of comments is LLM-generated and asking for skepticism.

It's not particularly helpful; you could easily have done the 5 minutes of work.

by b40d-48b2-979e3 hours ago|

parent|

[-]

Congrats, you've discovered people are always joining an anonymous platform, and some want pseudonymity? If I put "md" in my name, am I suddenly a doctor then? Your reading comprehension needs some work, I've been working for 16 years and it isn't being used as a cudgel to appeal to authority, but sharing some context about who I am and what my perspectives are.

by sarchertech2 hours ago|

parent|

[-]

Someone using md in their account name for 14 years is really committing to the bit if they aren’t a doctor.

I went back in their comment history before LLMs existed and found comments where they claim to be a doctor and sound like they know what they are taking about. I’m not a doctor but my wife and many of our friends are, so I know what they sound like.

by b40d-48b2-979e2 hours ago|

parent|

[-]

I appreciate your feedback, I did not dig through their comment history to make that discovery, but saw someone showing full trust in a random user on an anonymous forum and wanted to encourage more critical behavior.

by sarchertech2 hours ago|

parent|

[-]

I get the intention, but you made an assumption that the person you replied to didn’t spend 30 seconds find some more supporting evidence before they made their post.

But as far as trust goes, Hacker News has historically been a fairly high trust community. LLMs have the potential to change this dynamic, but I don’t think encouraging people to assume that every post is an LLM is helpful. I don’t think a community with that level of distrust is possible, and at that point we should just all walk away.

by nativeit3 hours ago|

parent|

prev|

[-]

The parent comment is demonstrating the exact type of internet savvy critical thinking that you’re ostensibly arguing for, but you seem to be reflexively defensive.

by b40d-48b2-979e2 hours ago|

parent|

[-]

Making an ad hominem attack on a random poster asking for critical thinking on who to trust and then making false claims is not "internet savvy critical thinking" since they can't string two thoughts together coherently (a bio blurb vs. an appeal to authority in a random comment).

by jmhmd4 hours ago|

parent|

prev|

[-]

Not taken negatively, and you're right. This is even more difficult with things that are opinion, and not clearly verifiable.