upvote
Just to be contrarian, perhaps some measure of risk is reduced by the scale of one.

Identifying a vulnerability that can be exploited against many thousands or millions of targets is perhaps more attractive than a single one of individually low value.

This of course would assume that vulnerabilities are in fact unique (which is admittedly questionable).

reply
To take this further, don't LLMs justify lowering the "barrier to attention"; i.e., if it only takes Claude's and not the hacker's eyeballs on the software, won't people find vulnerabilities in custom software for one too?

Besides that, one could easily imagine software created for similar purposes ("make me a file editor") by the same tool or handful thereof (claude and a very small "etc" for completeness) might share similar vulnerabilities, so this kind of broad net might be even cheaper to cast than one might imagine at first.

reply
> This of course would assume that vulnerabilities are in fact unique (which is admittedly questionable).

Yeah, I don't think all that generated software will be as unique as people expect.

Considering it will be generated with the same LLMs that all share roughly the same training data we will se patterns of vulnerabilities will also be similar and so easily exploitable.

reply
I had the exact same thought. Pretty low probability that there's going to be a script-kiddie exploit for your custom tools. Pretty decent probability that there will be vulnerabilities present if someone cares enough to target you.
reply
The counterpoint to that is that the exact same tools that are allowing this personal software creation at massive scale are also excellent at black box vulnerability analysis…
reply
There are entire vulnerability/fault/misdesign classes that are fairly general and appear to naturally emerge.

See e.g the lock screen gap that another commenter noted in a nearby thread.

reply
But the exploits can use AI custom tools too. "Script Kiddie" is just now "Prompt Kiddie"

Although everyone might use their own flavor of "database" or "REST API", I can't imagine every layout to be unique enough to not have similar exploit classes entirely. AI isn't known for being super original after all...

reply
Otoh, TAU will bound to get really personal now:D
reply
We should expect the same automated personalization to be used offensively and for that personalization to be packaged into tools anyone can run (natural language interface, likely.)

(Appreciate your counterpoint for its own sake. It’s an interesting idea.)

reply
If a vulnerability of the common not individualized ancestor software is found, how quickly do people patch their individual versions of the software?
reply
If they’re hosting network services, sure. I wouldn’t put vibe-coded software outside a home network, ever. But it seems low risk if people are just creating their own desktop software: especially since it’s less likely to be vulnerable to widespread malware.

(Note: I’m not an LLM fan, don’t vibe code myself at all. But I would be unconcerned about security for the kind of things I would create if I did start doing so.)

reply
But your browser will invite outside software into your network, to run on your machine. So you have to be up to speed with community knowledge.
reply
The article is about desktop software. If it does not accept network connections what is the risk? If it needs to do so you can run restrict it to you LAN or a VPN or over access it an ssh tunnel. If it replaces something you use over the public internet (e.g. SaaS) it might even be more secure.

Rolling your own might make you more vulnerable to targetted attacks, but less vulnerable to automated attacks looking for known weaknesses. Most people will not publish their code. The article says "It’s not an invitation to use my software. Honestly, please don’t. None of it is built for you.".

You can roll your own software and still use libraries for security sensitive things like encryption.

Even the author of this article (who is taking it much further than most people will) still uses Firefox, Weechat, and X11.

reply
Not everyone's "personal software" runs on a publicly accessible host on the internet.

I trust my Browser, OS and file system too.

But I'm also pretty sure none of the bespoke software I have will get any kind of security implications. The chance of my own file manager having a buffer overflow RCE triggered by a random file is practically zero.

reply
That seems like a naive view to me. Most modern software development is gluing vendor code and libraries into a CRUD app, and I don't see why that would change with agents doing the majority of programming. If anything, there's an even bigger market for solid libraries and interoperability, plugging things together like LEGO - only for real this time.
reply