undefined

points

[-]

Humans writing more like LLMs, just like new LLMs write more like humans, it's all coalescing into one.

I've copied-pasted comments I made on HN from like 2020 and had it tell me it's "100% AI". I've seen examples where the services claim "100% AI" because there was no normal dashes, only em-dashes. Even have a recent example from HN itself: https://news.ycombinator.com/item?id=48165690

> This reads very AI. Pangram [0] agrees [1]. [0] Not perfect, but I think as good evidence as any: https://arxiv.org/pdf/2501.15654 [1] https://www.pangram.com/history/44cd07d3-ba94-4331-8c7f-a626...

Said Pangram report literally citing the single evidence of em-dashes...

by albumen8 hours ago|

parent|

[-]

Your evidence seems very anecdata. The graphite.io study does make an effort to quantify the false positive and false negative rates of the three detectors, rather than just saying “they work”. They generate 2000 ai articles and ask the detectors to evaluate them, measuring the false negatives (articles falsely IDd as human written); and they use a separate pre-AI dataset (years 2000-2022) to determine false positives.

by embedding-shape7 hours ago|

parent|

[-]

Yeah, I suppose it is, I haven't finished my dissertation on it yet, I'll get right on that :)

Throughout them being available I've tried them every now and then, both with AI generated trash and my own pre-LLM writings, and had about 0% success in getting them to accurately report what it actually is. Maybe my writing style and what specific LLM you use matters a lot, I'm sure these platform's training data is mostly from the mainstream models so as soon as you use anything else, they'll get trivially lost. But again, I don't have any evidence and proof behind this, based only on when I've tried to evaluate them myself in the past.