Why should this be an AI system, and not just a software system?

You would also probably prefer a surgeon who forges ahead without it if the AI is down and you are going to die.

Claude is going to write a banger medical drama about this