By symbiosis I simply mean mutually beneficial coexistence.
We've barely achieved that with each other, but my point here is that we should strive to achieve that with the machines.
We're still treating it as if someone has to win and someone has to lose. Someone has to come out "on top."
If that's the game we want to play, against an opponent that is self improving... that doesn't strike me as an excellent strategy. We should probably pick a different one.
--
To answer your question, the incentive is so that we can continue to exist.
If the market doesn't reward our continued existence, then we should probably have a conversation about that at some point... I've heard the global economy described as a misaligned super-organism even before AI. AI mostly just makes it go faster in roughly the same directions it was already going.
> "If the market doesn't reward our continued existence, then we should probably have a conversation about that at some point"
---
There is a lot of "we should probably" carrying the point here.
The incentives are made by the market, and it happens to be an incentive that this discussion, or any action for that manner does not happen. Given the current status quo, and the foreseeable future given the incentives, how do you suggest that we even get to a time where symbiosis is on the table?
If you take antibiotics, your gut basically becomes a void (to simplify a little), and then it can get overrun with invasive species.
That's how I see what is happening with the internet and with AI.
When AI intrudes in a network, it's able to get very far because there is nothing there to stop it. There is no healthy ecosystem there. They're probably should be, and in the near future there probably will be. (Well it'll probably look more like a monoculture until we learn that lesson for the nth time, but... yeah)
Simulating the "human search method" in silico doesn't need to include specific personalities, but they do provide a bootstrap, just as we've bootstrapped current AI with human language.
For symbiosis, there may be an advantage to the physical adaptability of biology.
In general, it's currently unclear if biology is emulating what we're creating in computers, if computers are emulating what we achieve biologically, or if both approaches are imperfect emulations of a different ideal search method.
Humanoid robots have been a dream for centuries.
But a more practical robot would be like an octopus or something. It would just be like this weird ball of tentacles that rolls around your house. No one's gonna buy that, even if it's a million times cheaper to build.