I think SGX is a red herring in this context. You can find it risky or not, but it's (1) not material to Signal's E2EE property, and (2) materially better than email, which sends all contact metadata in plaintext, regardless of whether you try and encrypt your message bodies. Bringing it up in this context seems like a category error.