(prized.dev)
A few basic questions: how reliable is this judge since it’s based on an LLM? What additional measures can an admin or someone with more technical knowledge take to tighten this further if needed? Are the rules in or used by this judge visible to the users or an admin?
Don't take my word for it, watch the news in <6 months and we'll certainly see it get bypassed.
This posture is applicable while we ease out of what I'm calling the "golden era" (loose restrictions on access with limited consequence) but one of the reasons I am pedantic and bearish about solutions like this is that they too have a short lifespan that we can already estimate the expiration date of.
Initial talks with customers resonated more with the internal tooling idea though. It's flexible enough to support both.
We're earlier along than this product though so there's a fair few wrinkles in our product.
Separately I haven't yet seen a great security governance model for LLM integrations. At an enterprise level I'd like to govern rules such that, for example, if someone gives an LLM access PII information or proprietary data, then it shouldn't have access to a slack integration or the internet. Controlling this at the employee or team level doesn't make sense, as an employee may have reason to make separate use of both. From a security perspective I want them to be able use LLMs with different types of acccess, but not necessarily the same agent at the same time. Furthermore, they could ideally chain together agents with different permissions in specific orders. For example they could have a workflow where an agent can reach out to the internet, and then have a separate one that can read/write to slack, and then have a third that can interact with PII data. If they tried to wire together agents in the opposite order, it should get denied.
When I talked to Lovable team, they told me Lovable is used primarily for prototyping, not internal tools and they are having a rough time trying to change the use case.
Hope my experience helps!
It's so much more annoying to go drag boxes around and type out little JS snippits in every chart as opposed to just telling claude to make the chart.
There are a lot of operational needs that exist downstream from dbt models that combine data from the various source systems.
I think the moat for internal tools is not where you think it is. Also, i see self host option, it is a nightmare too, no one will do it just for internal tools now. It was valid before AI era.
I am also seeing non engineering staff actually building tools with claudecode, paas like railway and stuff. People have become more efficient with engineer tools.
Along the same lines. I got so fed up with Quickbooks self employed (god awful software and company) for my own business I wrote a replacement using Cursor. It tracks expenses (categorizing each in proper IRS categories) and estimating quarterly taxes. Uses Plaid to pull in business bank accounts and credit cards and has income and expense reports. It’s 100x better than Quickbooks.