upvote
> "If that's the game we want to play, against an opponent that is self improving... that doesn't strike me as an excellent strategy. We should probably pick a different one."

> "If the market doesn't reward our continued existence, then we should probably have a conversation about that at some point"

---

There is a lot of "we should probably" carrying the point here.

The incentives are made by the market, and it happens to be an incentive that this discussion, or any action for that manner does not happen. Given the current status quo, and the foreseeable future given the incentives, how do you suggest that we even get to a time where symbiosis is on the table?

reply
Well, our best bet is that we succeed in making the machines be less stupid than we are, and hope that they figure it out.
reply
So the machines should create the human-machine symbiosis for some reason, interesting... If I was the machine I would maximally want about the same symbiosis with humans as humans have with cows.
reply
There's another fun biology metaphor here which is the gut.

If you take antibiotics, your gut basically becomes a void (to simplify a little), and then it can get overrun with invasive species.

That's how I see what is happening with the internet and with AI.

When AI intrudes in a network, it's able to get very far because there is nothing there to stop it. There is no healthy ecosystem there. They're probably should be, and in the near future there probably will be. (Well it'll probably look more like a monoculture until we learn that lesson for the nth time, but... yeah)

reply