upvote

points

by 6gvONxR4sf7o18 hours ago |
comments
upvoteby soundworlds14 hours ago|
next
[-]
If I understand this correctly, Anthropic's argument is now "yes this will blow up some of your infrastructure, but it will be worth it"

The problem is that no one has been able to prove that it is actually worth the cost. That is a very fragile assumption.

reply
upvoteby daveshistory49 minutes ago|
parent|
next
[-]
It's Shrek logic. "Some of you are going to die, and that is a sacrifice I am willing to make."
reply
upvoteby alansaber9 hours ago|
parent|
prev|
next
[-]
This has always been the premise. They can't fix the fundamental problems with LLMs but they can continue to optimise them for IE parsing large volumes of data quickly
reply
upvoteby szundi11 hours ago|
parent|
prev|
[-]
[dead]
reply
upvoteby jon-wood9 hours ago|
prev|
next
[-]
Everything you do a risk/reward equation, you just don't usually see it drawn out quite so starkly. Getting out of bed in the morning carries a risk that you'll trip and crack your head on the floor. Crossing a road carries a risk of being hit by a bus. Eating food carries a risk of choking on it. The same is true in computer security. The only truly secure computer is one you don't turn on, and even that carries some risk of an attacker breaking in and stealing the storage from it.

Whether you agree that the potential harms outweigh the benefits in this case or not those calculations are always happening, so yes, I guess you're right. That is society in a nutshell.

reply
upvoteby vrganj9 hours ago|
parent|
[-]
But if you eat food, I don't risk choking. They want us to take the risk for their reward.
reply
upvoteby pixl973 hours ago|
parent|
next
[-]
But if I drive a car, You do run the risk of getting ran over. We can come up with any number of analogies of varying rightness and wrongness here.
reply
upvoteby daveshistory48 minutes ago|
parent|
[-]
And then there's a whole truckload of case law about liability that comes into play.

We haven't yet written those laws for "AI."

reply
upvoteby hombre_fatal4 hours ago|
parent|
prev|
[-]
What do you have in mind?

You're paying for their services to collect reward for yourself, but also deciding your own risk/reward when choosing e.g. how much access to grant Claude for any given task.

I guess there's the case where the more capable Claude is, the more someone else can use it to find vulns in your services while Anthropic collects their subscription money? But that is mitigable risk that you shipped regardless of what Anthropic is doing.

reply
upvoteby esikich17 hours ago|
prev|
next
[-]
Sure. You start a PC repair business. At first, losing a stick of RAM or frying someone's motherboard is super costly when you are doing 10 a week. But once you're doing 1000, that's pretty damn good and easily covered. When you have more tools, velocity, and whatnot, the proportions change.
reply
upvoteby altmanaltman15 hours ago|
parent|
[-]
Wouldn't you lose multiple sticks or fry multiple motherboards as you scale and do 1000? If you're frying 1 at 10, that means you're frying 100 at 1000. Your costs etc will scale as well unless you actually lower the risk/reward ratio, no?
reply
upvoteby kuboble13 hours ago|
parent|
next
[-]
I think the point is that at small scale a single accident poses a risk of ruin to your small operations.
reply
upvoteby chrncirurp13 hours ago|
parent|
[-]
> I think the point is that at small scale a single accident poses a risk of ruin to your small operations.

At big scale, a single big accident poses a risk to ruin your big operations.

reply
upvoteby enraged_camel10 hours ago|
parent|
[-]
No, it does not. Every large company eventually has a big accident. They survive because they have both the resources (e.g. to fight ensuing legal battles, or pay fines, or simply weather a hit to reputation and the resulting downturn in revenue) as well as redundancy, different types of insurance, and so on.
reply
upvoteby daveshistory46 minutes ago|
parent|
next
[-]
Companies of all sizes should have insurance to cover such scenarios. You need to get tradesman's insurance on your repair work, or you need to ask yourself why the insurance companies won't insure you.
reply
upvoteby zaphar7 hours ago|
parent|
prev|
[-]
They also survive because they invest those resources in some amount of mitigation ahead of time. They don't survive when they don't scale their mitigations along with the business.
reply
upvoteby truculent7 hours ago|
parent|
prev|
[-]
The point is that if you have a 10% chance of frying motherboard, at 10 a week, you might expect 1 fried p/w, but it could easily be more which may be catastrophic.

At 1000, the number of fried boards will be more predictable and therefore the risk to the business is lower, even if the long-run averages are the same.

reply
upvoteby solenoid093716 hours ago|
prev|
next
[-]
That's how decisions are made IRL. Risk/reward is a thing.
reply
upvoteby vrganj13 hours ago|
parent|
[-]
This is risk to us and reward for them though.
reply
upvoteby daveshistory45 minutes ago|
parent|
next
[-]
Many companies would say that's the best kind of risk-reward balance. For them, anyway.
reply
upvoteby alansaber9 hours ago|
parent|
prev|
[-]
Exactly. Though with inference cost they're still only making money on enterprise use.
reply
upvoteby heisenbit4 hours ago|
prev|
next
[-]
Limited liability makes taking unlimited risks a rational choice. AI ‚only‘ scales this corporate model up and compresses the timeframe to the next disaster.
reply
upvoteby andai17 hours ago|
prev|
next
[-]
Yeah I was thinking about Simon Wilson's "lethal trifecta"[0] in the context of OpenClaw style "general purpose" AI agents, where people just gave it access to their full hard drive, gmail account, etc.

I was thinking you can't make the chance of catastrophic failure zero (we still hear about "Claude deleted my home folder"), but you can definitely limit the blast radius.

You can't get the risk to zero. But the opportunity cost of not playing the game is rising. So you accept some level of risk.

My personal take here is "why screw around with containers and virtualization when a used ThinkPad is $50". Just give it its own machine. Then it can blow it up all it wants. (Or a $3 VPS, as the case may be :)

[0] The lethal trifecta for AI agents: private data, untrusted content, and external communication - https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/

reply
upvoteby barrkel10 hours ago|
parent|
next
[-]
Containment of the execution environment isn't really the issue. It's API tokens that were designed with coarse permission scoping so agents get more power than they need. The risk isn't that your machine gets hacked. It's that your email gets deleted, or forwarded to someone who uses it to break into your other accounts via password recovery.
reply
upvoteby KaiShips1 hours ago|
parent|
next
[-]
[flagged]
reply
upvoteby shivyadavus5 hours ago|
parent|
prev|
[-]
[flagged]
reply
upvoteby zaptheimpaler16 hours ago|
parent|
prev|
next
[-]
I tried the VPS briefly, it didn't really solve anything for me. The personal assistant agent is only as useful as the data & tools it has, that's where the real risk is. Separate box gives you isolated FS but docker also does that very easily.
reply
upvoteby jon-wood9 hours ago|
parent|
[-]
Docker is not a security boundary. It never has been, but given recent demonstrations of container escapes its even less of one than it ever was. If you want to properly contain a process it needs to be running in a VM of its own, or you need to accept that there's a risk of it escaping and ending up with more access than you planned.
reply
upvoteby e12e11 hours ago|
parent|
prev|
next
[-]
Wiping out a VM, server or workstation should not really be a problem - just restore from backup.

Silently corrupting files, that goes undiscovered until after backup window closes, and data exfiltration are the immediate, serious risks.

reply
upvoteby koolba17 hours ago|
parent|
prev|
next
[-]
> Then it can blow it up all it wants. (Or a $3 VPS, as the case may be :)

Just make sure it doesn’t have ssh access to any other machines!

reply
upvoteby chrisweekly16 hours ago|
parent|
prev|
next
[-]
Is a used Thinkpad really a viable part of your AI workflow? (And is that really a better solution than eg smolmachines microvms?)
reply
upvoteby altmanaltman15 hours ago|
parent|
prev|
next
[-]
> But the opportunity cost of not playing the game is rising

The opportunity cost of not using OpenClaw? I don't think it's that foundational yet that there is an opportunity cost to not using it. Most people have no purpose for a general-purpose AI both in their personal lives and at work, there is no sense trying out OpenClaw when you don't even know what it'll do.

reply
upvoteby charcircuit17 hours ago|
parent|
prev|
[-]
All of ecommerce is built on top of encryption with a non 0 chance of being cracked. The risk is much smaller than the benefit so people are willing to use it and then deal with whatever potential fraud comes from encryption being broken separately.

Technically a merchant could require meeting in person to exchange a OTP to avoid this and make it 0 but it is not worth it and you will get out competed by other businesses willing to take on a marginally higher amount of risk to unlock a lot of utility for the user.

reply
upvoteby keithnz17 hours ago|
prev|
next
[-]
but no matter what you do this is the tradeoff you are making. Different people have different tolerances for that balance, hence why I'm happy to watch people on youtube in wingsuits and not do it myself. Of course in this new AI world, quantifying the probability and scale of harm is hard/not fully known. We are trying to mitigate risks with AI, but who knows, could be one misstep away from plummeting off a cliff.
reply
upvoteby 17 hours ago|
parent|
[-]
deleted
reply
upvoteby xp8417 hours ago|
prev|
next
[-]
I’m a usual booster of AI (others have accused me of being completely in the bag for the clankers) and even I agree fully. These yahoos would clearly give Claude the nuclear launch codes or enough access to copy its full model into the wild if the supposed “reward” promised was large enough.
reply
upvoteby daveshistory43 minutes ago|
parent|
[-]
Hardly a new hypothetical scenario, that Wargames movie is probably 40 years old now.
reply
upvoteby 7e17 hours ago|
prev|
next
[-]
They don’t consider risk of ruin and that is where this calculus falls apart. The reward does not reduce the risk of ruin, which increases with blast radius. YOLO!
reply
upvoteby ronsor17 hours ago|
prev|
next
[-]
This is how humans weigh most decisions in practice.
reply
upvoteby Maxious17 hours ago|
parent|
[-]
[dead]
reply
upvoteby Frieren11 hours ago|
prev|
[-]
> the amount of harm they're willing to justify goes up. Feels like society in a nutshell.

Neocon society. Socialism is not like that.

reply
upvoteby pjc5010 hours ago|
parent|
[-]
Running into the problem that Americans are very bad at defining "socialism" here, meaning anything from social democrat to full Communism, but: there is a strong utilitarian streak in socialist societies that is also vulnerable to "the pain (for you) will be worth it (for someone else)" reasoning.
reply