It's hard to draw any conclusion from this study precisely because of this. Since 2024, we went from AI being able to do a few minutes of coding work to now a few weeks autonomously. That's like going from an intern to staff engineer level.

It's also important to note that it beat doctors in diagnosing in a way doctors do not diagnose.

Yes, the preprint of the same paper (https://arxiv.org/abs/2412.10849) was first written in December 2024.

Medical research moves. Very. Slowly.