High hundreds of thousands feels like the upper limit before it would show up in statistically noticeable changes in patterns of deaths in some demographic.
High hundreds of individuals would still be "one in a million fatal errors over a few years", which seems better than I'd expect given I've personally had ChatGPT tell me that Solanum nigrum berries were "black tomatoes" (they're not usually fatal, but are a bit toxic, and no I did not eat them).
Imagine one of the models that has "accidental-deaths-via-bad-advice" just slightly turned up, with the model-provider's intent being to kill 5% more people per year.