Why wouldn't it be desirable? Is knowledge beyond that a child can understand undesirable because the child can't understand it? I think not, same with anything AI figures out that we can't easily understand ourselves. If GPT-10 Quasar grinds tokens out for 6 months and out pops a warp drive, and even it's executive summary is difficult for anyone to understand, do we get out the pitchforks and burn the data centers or do we go "Sweet, we've got warp drives!"
replyWhy is it always a false dichotomy between two ridiculous extremes? Maybe inventing some dangerous technology benefits from human understanding for a bunch of obvious reasons, like human beings being responsible it goes well?
replyMaybe fix energy and heat on earth first. Fusion would be nifty. How's the AI progress there?
replyDo we know that AI can't invent a conceptual framework?
replyIf we give it a real problem to solve, it may just have to invent a new form of math to solve.
It might. It might not though. Would a 1300s superintelligence advocate for heliocentricism in the face of all institutional players advocating for geocentrism, or would it create the most refined epicycle model imaginable? I’d guess the latter.
reply