As someone who has been working on this space for a while (not affiliated with govbase) this is really hard. Between eliminating the sycophancy that seems baked into LLMs and dealing with generalized hallucinations - it's freaking hard. I spent this weekend trying to figure out how to get my system to stop telling me the SAVE Act would be fine because it doesn't say what the process for if birth certificate doesn't match current id.

No, I haven't found a good solution yet - I'm going down a rabbit hole of basically crawling the entire federal register for referenced legislation and then adding in an adversarial agent to see if that can spot gaps.

Very true. We're constantly trying to refine this and eventually plan on hiring policy researchers for a human in the loop but we just don't have the funding for that currently. We are trying to be transparent for how our scoring does work which you can read more about here: https://govbase.com/methodology

The biggest issue we have found, as you have mentioned, is just the larger context. For example (I don't think this is a real example and would need to check), the TikTok purchase deal could be ranked as an overall benefit for gig workers making content since the outlined alternative was a flat out ban hurting their income. So a deal going through, alleviating that alternative of a ban, in a vacuum is good. However, that ignores the larger context of where that option even came from and the surrounding political context around that deal. So we know the system isn't perfect right now and we're constantly trying to optimize to get the larger picture.