> There is no guarantee that the LLM will help you converge on anything.

Absolutely. The guarantee does not come from the LLM. The LLM is a simply an improved version of Google Search.

The guarantee can only come from a systemic application of epistemic discipline and reasoning, which is very much (smart) human territory.

Put it another way, I could make good decisions with/without LLMs, with some uncertain diagnostics as input. I would have to trawl through 50 papers myself, and it is possible that my decision arrives 5 years too late as a result. LLMs enable trawling and do some of the legwork in connecting the dots, but are ultimately only as capable as the orchestrating human.

The same goes for a human expert. There's no guarantee of convergence and you could eventually end up at "I don't know".