upvote
Much like with 1984, I think the promising/terrifying part is that it's not an issue with technology per se, but about a society that has somehow decided to allow Terrible Things to happen. (Often via too much power in too few hands with no accountability or alternatives.)

For example, imagine that there are 20 great search-engines around the world (who don't collude), and it hits rather differently.

reply
I think (armchair analysis) it's a combination of two things, one is that it's by choice - people enjoy using LLMs, they find it valuable and convenient. As for social media, companies and society itself successfully managed to gamify people sharing their personal lives on the internet, to voluntarily reduce their own privacy. Also because they still believe they have a choice in the matter, by e.g. not posting or sharing things.

The other thing is the boiling frog analogy, it wasn't a sudden "we're at war now so you get a camera in your house" moment (iirc 1984 skips over the transition to an authoritative state though), it's a slow, gradual progress. People got used to taking selfies, then applying a filter, then using facial scans for identification, then a cool app that puts your face on movie scenes and now the company behind faceapp and co has detailed facial scans of millions of people.

Europe tried to limit it via legislation, but that's a lot of after-the-fact policing and that's just Europe.

reply
Orwell also imagined a future where Speakwrites made everyone unable to write by hand, making their thoughts dull and tanged as a long unprepared monologue tends to be.

I don't think he's anywhere on your side.

reply
That sounds like AI though, no? I've seen many below-average writers that are now dependent on it to write even medium length passages
reply
Are they dependent on it, or just prefer it because they don't like writing?
reply
What does it matter? They're not writing on their own. All roads lead to Rome with this one
reply
To make the point explicit, Orwell wouldn't find most dystopian the fact your Speakwrite refuses to write anti-state messages, and support the right to subsidized local uncensored Speakwrites. The battle for the public minds was lost when they forgot how to write.
reply
Orwell imagined it 80 years ago, not sure it’s more relevant just because someone else imagines it today.
reply