upvote
Yes, mechanical question of "does this exist" is trivial to solve. Whether the research is actually aligned with the user intent / the AI makes claims that are substantiated by the research at hand are much harder problems.
reply
> I don’t think hallucinations are a major issue with agentic coding a year later? At least, not for me.

Are you still looking at code and agent traces to know if this is true or not?

At work, we still see hallucinations across all model families as we share bot claims for each others code

reply
An issue I come across is the attributed failures of AI of a first time use of AI.

Non-deterministic software can behave orders of magnitude different based on how input is structured.

This factor is largely missed by bringing a deterministic mindset to trying AI.

This means the same problem solution can work for one person reliably, and not another and both are true.

Assertions that begin with a deterministic assessment using a snapshot in time of both the user and models capabilities are a new feature of this new category of software that will have to be better caught and expressed.

reply