I don't care whether I'm writing code or reviewing LLM generated stuff, but the prospect of losing my job and having to survive on welfare for the rest of my life isn't nice.
What seems to happen though is it always gets pulled into a discussion of "But they can't X" or "but ma taste!" or the old generic canard "can't replicate what's not in the training data"
All I want is for people to see that yes, this is happening, accept it, then figure out what a good response would be to it. Instead we get everything from stochastic parrot parrots to "Dario is just marketing when he tries to warn us" to the old an thoughtless "But if you think it's bad, why are you doing it?"
Please.
Change doesn't happen until we are at the absolute brink of IMMEDIATE catastrophe.
I think you might be viewing AI and humanity as a zero sum experiment. It's really not. Go read some David Brin (Existence is a good start). We don't know all the positive and negative aspects to come, but we aren't in a dark forest situation. Existentially, AI is here, what are we going to do with it?
And I’m wondering if this isn’t the enormous amount of organizational debt from having security second to everything finally coming calling.
I’ll give you an example of one that was written recently actually.
The initial compromise happened because the app explicitly did not verify auth claims when a specific string was in the ISS field. Well, fuzzers exist and are common.
The next issue was that once you’re in, there was no delineation between admin and regular users. Everyone had all privileges if they just made the calls.
Anyway, we did the usual post-remediation investigation and write up. The devs were of course using the latest models, as they were instructed, and the issue stemmed from a problem they’d been having integrating a specific company into their auth scheme.
Eventually, after many enumerations, the model opted to just skip auth altogether if that companies ISS was present. The devs, being in the habit of just accepting the changes did so and because of the nature of the code implemented nothing caught it in the pipeline.
This is sadly an incredibly common story and it won’t be fixed by models improving I don’t believe.
And yeah, I agree, there is no bottom anymore.