I'm trying to do more than that. If we validate claims made by the content as anchored in the user's trust graph, we don't have to care which server it came from--it's trusted for different reasons. Are you saying we shouldn't try just because it's hard?
Trust doesn't scale, and the internet is something that operates at scale.
All of the systems you're talking about have already existed and been tossed by the wayside. People could always have exchanged things like GPG keys, and yet here we are in a world in which we don't.
The vast majority of your users do not want to manage trust. They want it to be taken care of. Almost no one gives two shits about a trust graph, except maybe nerds like you or me. Exposing the average person to these concepts doesn't increase trust. It just makes the world more confusing for them.
This starts to delve into the monkeysphere and how many 'close' relationships people can form. If we have just a few things we trust it might work. Once you get over 15 or so it starts breaking down, and at 100 it's out of the question. Again, trust works on our evolutionary monkey brain that is used to a small tribe and systems requiring personal trust do not scale past that.
Trust graphs are worthless when they are built on a house of security lies. It's easy to get kicked out of them, but nearly impossible to secure your systems to the point you can keep attackers from stealing them and pretending to be you.