This is a factor in favor of stability/security of software, but there are many others against:
- software (code) changes all the time, so there are windows of opportunity during which a bug is exploitable; in addition to that, a bug may take a relatively long time to be fixed
- a model used for attack may be stronger than the model used for defense, both in terms of model quality and compute allocated
- with software complexity increasing (and team/companies behind projects getting bigger), the margin for mistakes grows thinner, and introducing misconfigurations or weaknesses becomes exponentially easier (with "exponentially", I mean literally, because the interdependence of the components, both technical and human)
And last but not least: in general, attackers are more skilled than defenders; in best case, defenders are well-trained. And the idea of having the population of potential skilled attackers growing is very unsettling.
I could see it going either way.
If it requires a lot of compute and trying, this is something that could be provided for common software.
So the whole thing is forcing the good guys to outspend on tokens to preemptively defend against the risk of the bad guys outspending them on tokens, rather than buying tokens to actually add features to the product etc.
So are they creating a market for the solution by helping create the problem? A kind of rent-seeking AI security-industrial complex!!
White hats are constrained by needing to pay for their own tokens, only using (expensive) vendors who meet governance and risk requirements etc. Black hats are free to take over accounts and steal services from wherever they can.
The path to substantial profitability for Anthropic is questionable. The Chinese LLMs threaten them by far the most of the three major US LLMs. The money for Anthropic is certainly not in $20-$200 subscriptions. And they don't have anywhere near the consumer potential that GPT does, in terms of unleashing an ad spigot. So how far will the API money scale while being undercut by China.
OpenAI has to fight with Google for the ad business, they're specifically building Gemini to focus on consumer + search. Anthropic's business looks cute next to Google's search ad business (which is entirely at risk in this inflection). Meta looks like the biggest potential loser right now, ad dollars will be sucked out of the rotting Facebook network (not Instagram) and redirected to the rapidly expanding, hyper rich context LLM interaction. Advertising on Facebook will feel like running dumb banner ads on Excite in a few years, compared to what GPT will know about its users.
People that think Chinese LLMs are a general threat, don't understand consumer destination services, which is what GPT's future is. China currently has nothing to threaten with in that realm. There is half a trillion dollars of advertising up for grabs.
This is just not true, building an effective advertising platform costs significant amounts of money, time and people.
Remember that you need to hire a sales force for this, and sales scales linearly rather than sub-linearly like engineering.
Additionally, you need to spend a lot of money dealing with fraud, fake and malicious ads.
Furthermore, you need to figure out where to put the ads and how to rank them.
Finally, advertising is a zero sum game (given that the internet has already killed lots of print & OOH advertising), so the only way to win is to better better/cheaper (preferably both) than Google/Meta/Amazon. Best of luck with that (although to be fair to OpenAI they did hire Fidji who knows a lot of this stuff from her time at Facebook).
They don't have a Sheryl Sandberg type figure, and she was also really important in selling FB ads to large advertisers.
Just looking at their leadership team I don't see anyone with a background in (successful) ads companies, so I'm pretty sceptical that they can build this out quickly enough to matter.
This is one reason
> and trying
and this is the other.
As a matter of basic logic, there will never be a time when it will be known that there are no bugs.