https://nebuchadnezzar-megolm.github.io/static/paper.pdf
I don't know how much things have changed, but this is the paper's conclusion:
"We presented six attacks that together invalidated the fundamental security promises made by Matrix’ end-to-end encryption against a malicious server. In particular, the version of Matrix as implemented in Element and analysed here neither provided confidentiality nor authentication against such an attacker."
My weak understanding is that if you're using it just for internal use / between friends then it doesn't have as much effect - but in public rooms anyone with a server that joins can decrypt your messages.
I think Element is slowly (slowly) working on improving security.
https://element.io/blog/element-web-device-verification-beco...