It must be something to do with Microsoft being the owner now of GitHub
replyNow that's just speculation
replyYou know what? I had honestly forgotten about that xD. /thread
replyMy Lethal Trifecta talk at the Bay Area AI Security Meetup - https://news.ycombinator.com/item?id=44846922 - August 2025 (115 comments)
replyGood read thanks.
replyAlso interesting to see who coined the term prompt injection.
Yea agreed. LLM guardrails are either just written prompts as in "Please do not bad stuff :(" or other LLMs verifying that the first LLM didn't so some bs. Both of wich methods do not work sufficiently as time shows again and again.
replyFunnily enough, nobody expects quality software anymore and errors became tolerable. So thats a win (for someone like me that lost all passion for the industry).
Agree with your assessment of guardrails. They barely work on the best days. We need to flip the idea of “agent” on its head. The agent here is an agent of the user interfacing with GitHub. Not an agent of GitHub interfacing with the user. Prompts and guardrails cannot keep the agent loyal to the company. Stop giving these things any permissions the user doesn’t have, and recognize them for what they are: a different UI than web forms, but still the same security model.
replydeleted
replyThat last part is I think called negligence. And in some industries that becomes criminal negligence quite quickly.
replyMost companies I ever worked for inherently operate on criminal negligence, and even when addressed, have no interest in fixing it.
replyGuardrails are essentially part of the input. Saying "but we have guardrails" is like saying "but we do trust part of the input".
replyEither way, even if you trust 100% of the input, there is actually no way to guarantee that you can trust the output of the LLM. (Which, I guess, is also true for every dependency you pull in. But for those, you at least have ways to audit them.)
deleted
reply