The TEEs are real. The problem was caused by software bugs. Running a TEE app (CVM) is more than dropping a binary into TEE. Two problems remain: (1) migration, you don’t want to completely lose the data if the hardware dies, and (2) upgradability, you want to patch the app when there’s a bug without building the state from scratch. So we want to decouple the states from hardware.
Using the hardware key is painful. The solution is to introduce a separate key manager (KMS). It releases keys to the app iff. the app follows a predefined “policy” (code, hardware attestation, etc). We can decouple the states from hardware based on a remote KMS.
This is rather a common design. You can find it in our system, as well as other mature frameworks like Confidential Containers.
As the system grows, a bug was introduced to the policy of how we replicate a KMS’s key material (for redundancy). The attacker exploited it with multiple sophisticated attacks, eventually being able to cheat a running KMS to hand over the keys to a fake one, like it’s a legit KMS replica. As a result, the keys protected by the KMS became at risk.
To mitigate the problem, the legacy KMS was dropped and a new KMS was deployed. We have then rotated all the keys associated with the legacy KMS. A series of protocol and code audit is ongoing to prevent future failure.
(https://docs.phala.com/dstack/design-documents/decentralized... , https://docs.phala.com/dstack/design-documents/key-managemen...)
Actually it looks like a pretty decent solution once the inevitable bugs and footguns are ironed out.