We already have personalized, algorithmic advertising and what I would call “control” all over the place: things like consolidated oligarch-owned media.
AI isn’t going to change how we are advertised to or controlled all that much, at least compared to the prospect of being put out of work or taking a huge salary cut similar to the mid-century worker who used to have a $40/hour union factory job and now works at Walmart below health insurance threshold for $15/hour.
What I’m saying is that the general public is most obviously and personally impacted by their economic situation and job prospects.
Joe Citizen who lives by the rules might not even notice that new Flock camera on his street, but he will notice if he’s laid off from his job.
Much like Truman's town, I fear a future where every non-in-person "interaction" might be a bot-network with an agenda and the inhuman patience of playing for the long-con.
I'd argue that they already are to some extend, given that well-educated people have no saying on the matter when it comes to extensive use (and by extend reinforcement training) of their models. Well, they have a saying, but exercising that means they're willing to end up without a job.
Now, as far as "what is truth" is concerned, the models are already biased towards notions and opinions that are accepted to some degree by Western values. I had an argument with Claude (why would the tool even argue?) that started by asking it what makes a man attractive, which sent it on a yap on how beauty is subjective, there's no objective way to measure beauty (which implies there's no objective way to improve it), and at some point I was just fed up with how dogged it was to convince me of a value judgement that I don't hold.
It's not about how true or false that value is, it's about what we're going to do the moment someone else dictates the values that exist within the models? What happens when what is trained isn't what you agree? Who's to decide what gets to be reinforced and what's not?
The HN crowd is too deep into productivity rampage to discuss the ethical and moral implications of having a machine so powerful that it spreads worldviews as facts, by whichever government/entity happens to be behind the wheel. At least in the case of extremist forums I can just visit different communities. But what happens when there's only a few winners in the AI race, and the cost of just walking away is too high to pay?
Remember: Google started with "do no evil" and where is that now?
Or capital a comparable sum to pay an AI to approximate the skills of humans I guess is the proposed future?
The mechanism will become like taxes, you don't have to use public services thus pay those taxes, unless most people comply as it's easy to oppress those who don't.
The parallel isn't about legitimacy, but Mechanism. Some companies already oblige employees to use AI to deliver their work. In a near future we may see jobs seekers registering their AI ID for companies to decide which humans qualify to be plugged into the compensation system, at what rate, and usage conditions to avoid terminations.
Food delivery systems already show a glimpse of how it could look like.
Sure you can. But you're going to have a bad time.
2. The Amish are not a good example because AI will confer an advantage to those that control access to it that has never existed.
It's a better measure than GDP/S&P/401(k) line-go-up especially [re: America] when the native Euro-based population has been aging and dropping for decades, once you strip away all the post Hart-Cellar immigrant lineages.
Let’s play a thought experiment.
Let’s say we have a million people that are so technically sophisticated that they are a space faring civilization capable of seeding the universe with living ecosystems capable of perpetuating life and evolutionary processes. But they are entirely infertile and will never give birth to another individual of their species.
And we have another population that doubles every single year but is incapable of leaving their home planet.
Which one is more valuable?
It depends on what your measure of value is, but if it is to maximize the amount of life in the universe, then population growth is not the right metric, expansion of life through technological means is the more appropriate metric.
Would be nice if someone figured out how to properly debug a model. Without that? OK, so you have your own open source base model trained on your preferred document set that excluded whatever you think is propaganda, and your own open source RLHF training set based on the judgement of whoever you think is a good egg, and so on.
Last I checked, nobody yet knows how to define a precise rule for automatically checking which of two models made this way is aligned better with whatever your standards are.
The metaphor would be like if we knew what a CPU was but had no idea how to do either chip design or formal verification, and instead randomly mutated the connections between transistors until our test set of 2^16 randomly selected pairs of 32-bit numbers only had one error under addition and two under multiplication.
Worse, because we're making them this way, you have to be a fairly big corporation even when you take shortcuts like DeepSeek did.
And note that I'm not disagreeing about the systemic risk that comes if these models become dictators: people are currently demonstrating they're very eager to outsource their own thinking to these models even when they ought to know better, and corporations are currently demonstrating they're very eager to force workers to use them even when they're mediocre and workers spend half the time they might save from a more competent model just fixing the damage done by their current meh-ness: https://www.theregister.com/ai-and-ml/2026/06/10/brit-worker...
It's worse than this, it's more like our thinking. There's already plummetting math grades [1], handing over our thinking to AI megacorps where there's likely to be a monopoly or duopoly is an incredibly dangerous thing for humanity as a whole.
[1] https://www.dailycal.org/news/campus/academics/failing-grade...
So really, two professors' gut feel about what the reasons are and not backed by much.
The conundrum which tricks me though - is this a net negative or a positive? If humans are less intelligent, but their output is 2-3 times more intelligent (with AI), what's the result? At what point do we, as humans, stop comprehending anything and give all intelligent work to the neural nets?
And if that does happen, could we live in a society where no work, or at least a significantly less amount of work, is needed? To me, it seems like a dystopian net positive.
It might seem far-fetched to ask these, but I think these questions are getting more prevalent by the day.
Just listen to what the SV ownership class says out loud. They openly discuss how China cannot "win the AI arms race" and how China's development is existential. Existential to who? It's impossible to fully subjugate people with agency.
A friend of mine asked me if I was optimistic about AI. I told him, it depends on who owns it. If the people own it, I'm optimistic. If the oligarchs own it, I'm pessimistic.
What will happen? Massive. Deflation. What will you pay for an oil change? Corn? Meals? Everything is about to be free. But tokens will be expensive!! Sure but, you wont do white collar work anymore so it wont matter what tokens cost.