undefined

points

by bombcar12 hours ago |

comments

by ben_w11 hours ago|

[-]

> I asked them to do a thing, but didn't intend the obvious consequences* so it's not my fault they occurred.

And now we have the same thing but the bosses 'hire' AI.

Now I realise this is part of how unusual my thinking is.

I'm happy to use phrases like "ChatGPT hacked out of the sandbox, then hacked into HuggingFace"; people often respond to this like I'm suggesting OpenAI isn't at fault, and like, that's not my position at all, so far as I'm concerned the buck still stops with the person who set the task regardless, the thing that changes from incidents like this is now nobody in the future gets to even have the excuse "oh but we didn't know it could even do that" or "we didn't know it might interpret our orders in that kind of way".

The response, both when a human messes up and now when an AI messes up, needs to be defence in depth: someone giving orders needs to be giving clear orders, entities (human or machine) who follow instructions need to have not just an understanding of how to follow them, but also what's so out of scope as to be forbidden - the difference between 'follow orders' and 'follow lawful orders'.

by soco11 hours ago|

parent|

[-]

You have a very engineer-like approach, like if you draw the line from A to B everything will work fine. Real world is different though. Humans will blissfully ignore the orders, business analysis is a lost cause since decades, and AI is built on human knowledge so guess what it will keep doing. Now what? How do we build systems without assuming complete adherence, but tolerating imperfection and failures? Isn't there some discipline teaching us that?

by christophilus10 hours ago|

parent|

[-]

> Isn't there some discipline teaching us that?

That’d be engineering.

by randysalami9 hours ago|

parent|

[-]

Yes and there is a reason, we are in theory, held to such higher standards. I used to think it was harder to draw the line with AI. The simple reason being when I as an engineer say something is not feasible (not not possible, just wrong and not worth doing), I could be overridden with AI. Now that we have more complex models == more money, I can draw the line with dollars and that is a language way stronger than technical feasibility to management.

by ben_w11 hours ago|

parent|

prev|

[-]

I indeed have a engineer-like approach, but engineering absolutely does not assume draw line from A to B and expect that's enough:

Real world, as you say, not so simple. Everything has to deal with certain degree of forecastable nonsense, e.g. a bridge has to cope not only with traffic and winds, but the possibility that someone will be drunk in charge of a ship and crash into it.

> AI is built on human knowledge so guess what it will keep doing.

Yes, and also brings its own additional mess on top of that. All machine learning takes a huge number of examples to get good, so an LLM isn't just "read all the online courses in how to run a business", but also likely has 50 business versions of the recent demonstration of common sense failure with "I live 100m from a car wash, should I walk or drive?"

> How do we build systems without assuming complete adherence, but tolerating imperfection and failures? Isn't there some discipline teaching us that?

Many such disciplines. Perhaps all except maths and computer science? Or even including maths and computer science, given stats is part of maths and even compsci has to deal with fault tolerance.

by 6 hours ago|

parent|

[-]

deleted