undefined

points

[-]

A separate model with separate context is used for review.

Like I said above, some people will never be happy with LLMs being allowed to do anything and nothing is going to make them happy about it.

It’s only fair to discuss what the real current status of these systems is. Every time I highlight that things are actually being done, the goalposts move again. There is no possible solution which will satisfy someone who has zero tolerance for letting an LLM execute tool calls because they will always find something.

by acedTrex6 hours ago|

parent|

[-]

> A separate model with separate context is used for review.

Thats fine, theres still a chance it fails.

> There is no possible solution which will satisfy someone who has zero tolerance for letting an LLM execute tool calls because they will always find something.

This is generally correct, security goes completely out of the window with this stuff. It will/currently is a security disaster and theres no actual solution to it.

by 5 hours ago|

parent|

[-]

deleted

by wang_li7 hours ago|

prev|

[-]

"Let's make sure our model fails sometimes so that we can bill more for a second agent to validate, sometimes correctly, the work of the first model."

by Aurornis6 hours ago|

parent|

[-]

If you’re implying that the LLM companies are trying to train their models to make malicious tool calls so they can collect a few more tokens on the review, then I don’t know what to say. I guess threads like this are just a breeding ground for conspiracies now?

by wang_li6 hours ago|

parent|

[-]

I think that solving a problem they have that has a side effect of generating more revenue is not lost on them. Particularly in the situation where the cost to improve the models goes vertical for minimal improvement in an asymptotic fashion. The decision to spend 6-7 figures regularly on training or bill every customer 3-4 figures per month isn't hard. While I don't think they will explicitly impair the models, they will certainly make choices that externalize the shortcomings of the model to their customers.

by Aurornis6 hours ago|

parent|

[-]

> I think that solving a problem they have that has a side effect of generating more revenue is not lost on them.

I think you’re overestimating the revenue generated by this. Having a separate LLM with a cached input prompt check commands is a trivial adder. The only reason it comes up is because they explain to users that it comes out of their plan. So someone on a $20/month plan is going to hit their limits marginally, though mostly negligibly, faster.

If you think they’re sitting in a conference room scheming about making their main models worse on purpose to collect a few extra cents, that’s just baseless conspiracy. They have more to gain or lose based on main model performance.