There is a selection bias here. Not saying it wouldn’t work, but right now you hear about exceptional cases, not when the LLM wants to amputate for a wart.

We all work with LLMs, right? It hasn’t been long at all since an LLM gaslit me while attempting to recover an unbootable laptop. I should have been recommended a few simple steps to try; instead, it was unable to ignore the irrelevant details and led me on an hours-long chase. To me that means the LLM will also struggle to ignore irrelevant medical information.