The key here is that animations happen outside the foveal area. Our vision is tuned to be extremely sensitive to motion and changes _outside_ the foveal area. So when something moves at the corner of your vision, it distracts your attention from your current focus.
This makes a lot of "modern" UI literally anti-productive. It actively _slows_ _down_ people and increases cognitive load.