upvote
Humans seem to struggle conceptually with the liminal space between selection-pressure automata (an RNA virus; an insect which evolved camouflage), and an evolved complexity with agency, capable of "understanding" its actions (certainly humans; arguably many animals). And this makes a certain sense: we evolved to treat agentic things as being categorically different from non-agentic things. If we see sudden movement, it's vital to rapidly assess the difference between a tree branch in the wind, versus a predator.

The evolved complexity of the corporation seems to fit within that middle space: more sophisticated than a stick-bug (which exists merely from non-stick bugs being eaten), but not quite to the point where OpenAI/Anthrophic/Google/etc can "understand" its actions. And yet those quasi-intelligent feedback loops, evolving from iterated selection pressures of markets and ROI, seem to already be sufficient to domesticate us in their own interests, piggybacking on the nervous systems of employees, investors, customers, and citizens.

reply
Language as a SCP.

Remember when "The pen is mightier than the sword" was a popular phrase? Language has always been powerful. We know what it can do, why do you think every totalitarian government wants to limit it? But in our carelessness as humans we packed up all the language we could find and stuck it in an alien and now suddenly half the people on the internet are like "Don't worry, it can't do anything, it's just words".

We've made infohazards real.

reply