And as for street thugs, sure it won't be a wrench, more likely they'll flash a knife and unkindly suggest you remove the lock screen
Taken as a metaphor rather than a literal wrench, you don't think it's accurate?
This could be preferable to handing over private data about contacts, communications, sources, client information, etc. Especially if it has life-changing implications for yourself or other people!
In the wrench attack you are aware that you have been attacked, and you're aware of what data the attacker gains.
Additionally there are schemes like deniable encryption which can mitigate the outcomes of such an attack or serve as a red herring.
Furthermore it's dependent on physical intimidation which is expensive to scale and can be met with your own physical intimidation. In order for a wrench attack to scale to an entire society you have to send the gestapo to everyone's house whereas push-button attacks scale by default unbenownst to the victims and enable more nefarious systems to be built on top of them. In the USA this would mean interrogating some well armed citizens.
Lastly, you aren't forced to give up the key by any means. They can torture you to death and there is nothing they can do if you don't want to give up the key. There are some secrets in the game of love and war that are worth taking to - it's why spies are equipped with cyanide pills.
In a perfect society, your point makes sense, but I don't see why the authorities in the real world would need to care about committing a worse crime.
It works for the same reason locks on houses work against cops or criminals, despite the existence of lockpicking and locksmiths. There are various layers of physical security, and while no layer can prevent an attack absolutely they each increase the cost of an attack.
The system is only as secure as it's weakest layer.
So in a mixed information/physical system like a smartphone why should we allow the weak point to be the information system? To improve the information system we need only to rewrite the software, so the per-unit cost is nothing in the large.
The offenses of a regime at its apex would've led to it being stopped had they started out that way, but they didn't.
What you've hit on is the basic problem of treating privacy as a means to an end though: no level of it protects you from fascism, but it is a means by which fascism can be opposed - in many cases at personal cost to yourself.
In theory I have no secrets, and the contents of my phone or life if public would be of no consequence to me. In practice, when the regime starts flustering itself that I have no secrets for them to reveal the hopefully people will oppose it - or I get a decent warning that it's time to bail.