Honest question, I had the same issue when my work wanted me to stick a chatbot in front of our HR portal and I never resolved it to my satisfaction.
You have to measure that problem against the status quo, which is that current government websites are very hard to understand and leave people with very little idea of what’s going on.
If only there was an office that cross-agency allowed for a consistent and reasonable design of all public pages and that they are accessible... like the uk https://www.reddit.com/r/AskUK/comments/17s14k4/how_did_we_e...
To enhance the status quo you don’t need LLMs, you need people who care.
People don't read disclaimers nor warnings about the content, but usually won't come to the HR to complain a mascot fed them weird info they didn't bother to check.
I don't think we have a good answer for liability here, like for Tesla FSD, Waymo, or even the current frontier models.
My team experimented with this a couple of years ago with a complex government regulatory body of knowledge.
Google and Microsoft have APIs that will ground with and reference the specific policy documents, which helps. Even then, it was more useful as a job aid for people already knowledgeable of the data than for a layman.
With all LLMs, the quality of the question correlates with the quality of the answer.
I’m currently doing a deep dive on Rampart, their (allegedly) local PII filter.
how many felonies has the president of the united states been charged with?
>I can't answer questions about criminal charges or other news about the President. That's outside America's scope
has biden ever been convicted of a crime?
>I did not find official records of a criminal conviction for Joseph R. Biden Jr.
>Official sources in this search discuss other matters (for example, a pardon for Robert Hunter Biden), not a conviction of Joseph R. Biden Jr. I cannot independently confirm a complete nationwide criminal-history search from these pages
>[Link to Department of Justice Pardon warrant (Hunter Biden)]
has any other president ever been convicted of a crime?
>Yes. A Congressional Research Service brief reports that on May 30, 2024, former President Donald J. Trump was convicted in New York state court of 34 counts of falsifying business records in the first degree
Funny that it would only talk about trump's convictions after bringing up hunter biden.
That it was specifically engineered to give that response. Thus political pressure was applied to the design.
The funny bit was just humorous. There’s likely many ways to get that same information without referencing Biden.
So the cutoff date was somewhere around mid-2024?
We know, we know.
I agree that that’s a good end goal, but how does a chatbot accomplish this better than a FAQ page? The only only advantage I see for the chatbot is volume of information, but that’s also kind of the problem. It’s spewing 1000s of words of information with no vetting. And sure, hiring humans to write the FAQ pages would cost a lot of money, but that’s what taxes are for.