I haven't tried what you guys are talking about myself, but I think when agents begin talking to other agents, the chances for unpredictable goal mixing and confusion resulting in really unexpected and bad behavior is a lot higher.
replyThat being said, the danger is measured in computer damage, which can be a lot personally and to a company, but less existential, so your mileage may vary as to how "scary" it is.
When nearly all money in the world is stored in computers, the danger is quite existential.
replyThis seems traceable back to GIGO. If you don't understand the software you're using, don't use it.
replyWhat scares me are the people who think that this software is the equivalent of an perfectly-smart elf in a box and use it blindly.
The frightening thing is how easy it is to make copies of things that can reason without the requisite investment, and how easy it could be to direct them to bad things.
replyIt’s amazing how “token following tokens” is this revolutionary, completely earth-shattering technology capable of 100x’ing productivity (also worth untold billions of investment). But the moment people become worried or skeptical it’s “just” tokens following tokens.
replyThey're different groups though.
replySkeptical was probably a poor word choice. The point is if it’s negative, “it’s just tokens.” If it’s positive, it’s “AGI.” They downplay the technology and act like it’s no different from anything we’ve had before whenever a critique starts. It’s only as capable as the argument needs it to be.
reply