undefined

[-]

Your first link seems to 404; not sure if it's a typo or if the page doesn't exist anymore, but hopefully you'll read this while you're still in the edit window and can fix it

by cormorant1 hours ago|

[-]

Corrected link is https://www.theverge.com/ai-artificial-intelligence/975017/a...

by taude1 hours ago|

[-]

I think it's more about the mean. Worse writers, and thinkers are likely elevated by AI, and more impressed with the writing output. Decent writers and thinkers, are dragged back to the LLM-s mean of output.

by danggggg11 minutes ago|

[-]

[dead]

[-]

As someone who always has felt that I struggle to infer what people mean compared to the average person, I could tell pretty much from the first moment I encountered LLM-generated text that I was not going to be particularly good at recognizing anything but the most blatant and obvious examples. Pretty much anything short of a bunch of references to "load-bearing seams" or similar canaries, I'm always at a loss when seeing people argue about whether something is AI-generated or not because I can never tell.

I have no idea if other people who work in tech are better than average or not, because I don't feel confident in being able to check their work. That being said, I do think that there's a general trend of people in tech tending to be a bit overconfident in how well they will do at some new task they haven't encountered before, so when someone tells me that they can easily tell whether text is AI generated, it's hard for me to trust it any more than I trust someone who makes a similarly strong claim about something that they can use AI successfully for when it's not something that I can easily measure (e.g. learning a new language without getting feedback from people who are fluent from real-world usage).

All that being said, I do think the set of people who care is larger than just those in tech, although it's probably still a relatively small group overall. From conversations with people in other domains, there are contingents in non-tech communities who tend to have a large representation of negative views towards AI (artists, writers, musicians, other jobs where people are skeptical of human creativity being replaced by AI), and often times the people who feel negatively in those groups will be even more adamantly opposed to interacting with any AI content than people in tech. To be clear, I'm not at all trying to generalize and say "all artists hate AI" or anything like that, since there's obviously a wide variety of viewpoints within any sizable community, but I've definitely seen many people who say they will refuse to play any game that's suspected of using AI for generating art assets, and even some who don't differentiate between using AI for generating assets versus code (either because they aren't knowledgeable about how different aspects of game development work, or they genuinely don't care because they view AI as a categorical evil).

by wolttam1 hours ago|

[-]

Certainly not. Anybody who cares about language to any reasonable degree surely notices and is repulsed by heavily AI generated content.

[-]

I care about language a lot (feel free to go back through my comments from the past few days; you'll see a number of comments I made in debate about two different forms of a specific idiom because I have strong descriptivist opinions), but I genuinely struggle to identify whether text is AI generated. Maybe you're using "heavily" as the load-bearing part of your claim (sorry, I couldn't resist, another example of me finding language fun!), but I think you might be assuming a bit too much about how similarly others experience the world to you. A huge part of why I care so much about language is because I've always had to put a lot of effort into learning how to communicate well with others, and that ends up causing me to think and read a lot about stuff like how people use certain words in certain contexts to mean different things; the reason I care is pretty much the same as the reason I struggle with recognizing AI content.

by wolttam1 hours ago|

[-]

I think I might have been a bit heavy handed in my comment as I was rebutting the idea that only tech people can tell. I suspect it helps to have been exposed to a lot of earlier model writing, which was even more sloppy and had more of the kinds of tells we still see today.

And I’ll concede on both ends that there are probably times I suspect content is AI generated when it isn’t, and times I suspect it isn’t generated, but it was.

AI tells seem inevitable. You have millions of people communicating with one effective “personality” that has tendencies to write in certain ways. If its content is published verbatim, then it will be easier to tell whether some content is AI generated just based on its similarity (sharing certain linguistic features) to other content being posted.

It’ll never be black and white though.

by datakan1 hours ago|

[-]

> but people who spend all day doing jobs that don't involve computers aren't as good

I think they may just be to trusting and/or naive. People in tech right now are hyper aware of this and are actively looking while people outside of that bubble barely give it a second thought.

by legobmw991 hours ago|

[-]

I partially think the difference is “can you tell something is the output of Claude without any real prompting”. People can absolutely use LLMs to generate text that I wouldn’t recognize, but people who don’t care and are producing slop with the major models set to default settings leave these incredibly obvious signatures behind

[-]

Although you're referring to prompts given the Claude rather than the people attempting to recognize, it occurs to me that most of the discussion I've seen around people recognizing AI seems cover contexts where the reader is actively suspicious about whether content generated to begin with. Rather than a binary "is this text AI generated or not", I wonder if it would be harder for people to do a Coke/Pepsi style challenge where they're given two pieces of text where it's not guaranteed to be exactly one LLM-generated and one human-written, but they could both be from an AI or both be from a human.

Going further, I'm curious about whether people are mostly good at the case where they suspect most or all of the content from given "author" has the same amount of AI usage/prompting in generating it rather than the adversarial case where someone might usually use AI extensively and then try to slip by purely human written text (or vice-versa). I don't have a good sense of whether this is a threat model that actually matters, since maybe the heuristic of weeding out sources that are mostly AI-generated is enough for people who prefer to avoid that type of content, but I do think that changes the definition of what it means to be "good at recognizing AI" in a meaningful way. It seems plausible that disagreements about how easy it is to recognize AI content might be coming from two people assuming a different framing of the question that results in a different answer without realizing that's what they've done.

by legobmw9915 minutes ago|

[-]

I suspect both may be interesting to study more!

Several existing studies I’ve seen have done things like prompt the LLM to produce a poem in a certain poets style, then ask people to spot the fake in a collection of poems, which they aren’t great at. This is, I would argue, an extremely different context than what most of us are encountering AI text in, and the people sending me text aren’t prompting it stylistically like that.

On your second question, I definitely feel like I can tell the first time a coworker sends me AI text masquerading as their own thoughts, even if they had previously been opposed to such a thing. So it could be that familiarity is more important than my prior on whether they’d use AI? But interesting to think about either way

by eloisant49 minutes ago|