If you're talking about the baseband, AFAIK it's already isolated on both iPhones and pixels. Not sure about other androids.
Can you get arbitrary code execution on the RTOS from another app? No? Then adding layers to protect apps from each other is meaningful.
What you're saying isn't too far from "Well, if the attacker has physical access they can just freeze and decap the memory to read all secrets, so like there's no point in even hashing passwords or fixing XSS"