There's not that much downside since the annotations only change the performance characteristics of the program, and the static type system rejects inconsistent annotations.
I don't even have good conjecture about why this is the case but right now all my assisted coding is in MLs for this reason.