I think the point is that you stop caring about the fiat part of it at all. Government ownership of fiat currency is not a natural feature of the universe, it's just something we did for a little while.
Only if you wanted to turn it into fiat currency for the transaction, right? If the yacht seller would take payment in crypto, you need no government’s permission, right?
Surely the government would get in the way there, but holding cocaine is also illegal, not just transacting with it. If the government went that route, they'd have to ban all crypto outright.