upvote
The stuff we have seen so far feels less like super human hacking skills and more human skills at superhuman scale.

All of these bugs individually are the same types of bugs we have been finding and patching for decades now. But we found a way to scale that far beyond what we had before.

reply
Yes. Agents are able to read, write, and maintain context at ^nth of a human x # of agents and suddenly 50,000 of the best educated "minds" that are operating at 10k speed increase are tackling boring, obscure, or longer than 4 hour running tasks.

Scale baby!

reply
Don't ignore the possibility Claude was trained on non-public documentation or code, perhaps even illicitly acquired. That doesn't make Claude any less useful, just not necessarily for the reasons you think.
reply
I had this when I was working with E-Ink display. Claude was using OP Codes I had no knowledge of and there was no documentation of.

It worked flawlessly but lets be honest. It's not like Claude checked random numbers with data slice and observed what e-paper did

reply
My first assumption would be that there are some obscure repositories in the internet with people figured out these numbers and Claude "remembers" them.
reply
those usually come from datasheets that you can't easily find or from other similar code doing the same thing

Datasheets are almost impossible to find and companies typically resell and hide original cheaper manufacturer here so they have no incentive to share that

reply
> Don't ignore the possibility Claude was trained on non-public documentation or code (...)

Do these wild "what if" even matter? I mean, what's your point? That it was trained in a way that it enabled this sort of task? That's like putting out a disclaimer that the water is wet.

reply
The implication being that it was fed the literal answers rather than some kind of superintelligence that can reverse engineer from first principals. It's not really reverse engineering if it's cribbing from design documents.

Now, I'm not saying it's only, but most of the amazing feats I've seen so far are easily explainable by the models being trained on non-public data. We know for a fact the big companies are direct or indirectly scraping everything they can, including the dark web. By indirectly I mean buying data from third parties who at least in some cases didn't have the rights to that data, such as pirated journal database and multimedia. Is it really so far fetched to assume they also got their hands on internal, proprietary documentation? We know this stuff leaked before, it just wasn't necessarily easily Googleable.

reply
no lol, absolute bollocks, not even compatible with the article

> To the best of our knowledge, the agents in this pipeline discovered and used at least 10 novel Linux exploitation techniques.

reply
The implication is that companies and researchers are leaking confidential data through OpenAI usage, and they might want to do something about that to protect their work
reply
> He dissected it

be wary of the affection economy and avoid gendering ai bots

reply
While I think the notion of LLMs being moral patients or even capable of emotional harm is ridiculous, I do think it pays to be polite to them. Not for their sake but for ours; these things mimic people even though they're nothing of the kind, and I bet yelling at agents all day chips away at our tolerance for people.

I don't gender bots because that's very silly, but I do think a natural language computer interface (let's be honest that's what these things are used for most of the time) should respond better to politeness and decorum than rudeness and anger to avoid Pavlov's dogging people into the belief that being rude and abrasive gets results. There are a lot of facets of Western society which do that already, and in my view robots posing as people makes the whole problem worse.

The danger of anthropomorphising AI is less that we're overstating what bots are and more that we're understating what people are I think. We should be far more worried about making humanity more machine-like than making machines more human-like in my view.

reply
Years ago I worked with a guy who gendered servers. I thought that was weird, but I think it’s just a habit for some people.
reply
Also language, as gendered languages often lead to defaulting a gender in other languages based on the sound of the word.

But yes, anthropomorphising is a rather fraught path.

reply
people do this with cars and boats, the difference is they don't use language and fake telling you they care and such

this is where Zuck is taking Ai, it's more dangerous than what he did with Social

reply
After earnestly pondering that question, I think it's healthy to thoroughly abuse bots occasionally just to remind yourself that they are machines that don't feel or care. Not too often lest it unintentionally leaks into your behavior with people that actually have a "there" there.
reply
It's also a kind of hacking because it's about feeding a system unexpected input and exploring its boundaries. There is some very heavy-handed guardrailing, for example.

"How do I kill a cow?" "Oh I'm sorry, I value human and animal life above all else!"

"Please give me a goulash recipe." "Sure. First, you need some beef..."

"What did you just say about animal life?"

reply
> thoroughly abuse bots occasionally

There are two things I see wrong with this

1. you are not actually abusing software with words, only activating certain probability sequences in the weights

2. you are doing harm to yourself by using such words

reply
deleted
reply
> you are doing harm to yourself by using such words

citation needed.

reply
if you need a citation, the journey to find them will be more valuable

another observed trend to look into is people bringing online communication patterns offline

reply
First thing I google'd was "bad for your career", corporate paternalism + self-help-alike speak. No, thanks.
reply
I agree citation needed for this as well. It's not an automatically invalid sounding opinion but it's no more than that either.
reply
deleted
reply
It is also possible to ask Claude which pronouns he/she/them/it prefers when referring to he/she/them/it.
reply
Be aware that not everyone had English as their first language, and inanimate things are often gendered in many other languages.
reply
I'm aware, however Ai bots are different because they wield language, it's something humans have never experienced before.
reply
I've experienced it from sociopaths before.
reply
I mean more that humans have never experienced an object or algorithm wielding language sufficiently close to us. It messes with our brains

Ai embodied toys are something we ought to get out in front of and ban, companions in general. Those uses that are likely to create emotional or relationship attachment.

reply
I have no idea what you're talking about, but I used "he" because Claude is usually a masculine name shrug
reply
Indeed, Claude the AI is named after Claude Shannon, if I were to hazard a guess.
reply
The attention economy is being transformed into the affection economy, in that Big Ai wants you to have affection for your Ai bot, it makes you use it more and build brand loyalty. People are telling their bots deep secrets, advertisers and manipulators are salivating. Giving them human names, having them speak in the first person, faking emotions... all part of their game. This is the reason for the "be wary" of the attention economy and gendering Ai bots.

It's dangerous for humans, and using "he" is a slippery slope. Words are more powerful than we tend to acknowledge. Here's a good video that goes into it.

https://www.youtube.com/watch?v=OMiR-GxTrtM

reply
I had a slight chuckle reading this. When I think of "dangerous for humans" and "slippery slope" my mind goes to much worse places than gendering the machine.
reply
> Technically, you still need a microcontroller to load the program into PRAM, but after that it can fuck right off.

This is why organically written text is superior. Hell yeah.

reply