upvote

points

by mohsen15 days ago |
comments
upvoteby Chance-Device5 days ago|
next
[-]
I was wondering when something like this would happen. I got my first and only two content violation warnings in Claude Code last week when asking it about something ML related. It was a real head scratcher because I couldn’t figure out what about the requests could have violated anything.

Might be worth going back and taking a harder look at what I was asking it about if it somehow triggered a “forbidden knowledge” alert. Or maybe it was just a random bug.

reply
upvoteby throwfaraway45 days ago|
prev|
next
[-]
"for example, on building pretraining pipelines, distributed training infrastructure, or ML accelerator design"

Oh man all of those runaway infrastructure buildouts by our agents trying to achieve singularity...

Just say you don't want to lower the bar for others to compete

reply
upvoteby properbrew5 days ago|
prev|
next
[-]
> frontier LLM development

This seems so wide reaching if it's catching simple things like explaining a paper. Does this also refuse to help with any already developed training pipelines?

I can kind of understand the generation of synthetic data, but nerfing the assistance of training pipelines just seems like a really shitty thing to do.

reply
upvoteby alden55 days ago|
prev|
next
[-]
So insane to me that these ai companies are perfectly fine trying their absolute best to automate as much knowledge work as possible but as soon as this capability can be turned on them they start implementing hidden interventions to sabotage anyone trying to beat them at their own game.
reply
upvoteby gunsle5 days ago|
parent|
[-]
Not to mention these models are all built off of clearly extremely illegal abuse of human created content, which they will apparently never be held accountable for.
reply
upvoteby elastic-hoover5 days ago|
prev|
next
[-]
I wanted to try on my biology research and it refused to talk about it and proxied to 4.8. Really, only surface level conversations about topics of interest. I know this is not a topic of broad and mass interest, but limiting it for topics like that and machine learning will probably do change how I use it.
reply
upvoteby teliosix5 days ago|
parent|
next
[-]
I feel like it will have to become more finely tuned on topics of biology.

It is not just biology but is defaulting back to 4.8 for me on time series/information transfer techniques that happen to mostly have papers using the technique on neural data. Other information transfer techniques are perfectly fine, even cutting edge ones, but this one happens to be new and happens to only be discussed in terms of neural data so that is a no go.

With that said, I think it is absolutely awesome. The usage is really not bad at all compared to what I was expecting.

reply
upvoteby elastic-hoover4 days ago|
parent|
[-]
Interesting. This take of limiting ML and some science topics is worriesome. It's really nice to have a tool like that to help the research.
reply
upvoteby lxgr5 days ago|
parent|
prev|
next
[-]
Yes, this stuff is really annoying when it misfires. I've had all my subsequent ChatGPT conversations biohazard-contained for several days for the crime of asking it to explain a gene drive to me.
reply
upvoteby elastic-hoover4 days ago|
parent|
[-]
I've had all my conversations about bioloy denied. Even if I send a simple message containing only "Human" it gets flagged.
reply
upvoteby calf5 days ago|
parent|
prev|
[-]
Is it certain or all advanced topics? I'm curious if it bans questions about quantum computing or fusion.
reply
upvoteby elastic-hoover4 days ago|
parent|
[-]
It seems to be biology and cybersecurity
reply
upvoteby foolserrandboy5 days ago|
prev|
next
[-]
This is just marketing that Anthropic is building the singularity.
reply
upvoteby __blockcipher__5 days ago|
prev|
next
[-]
Anthropic is really speedrunning their evil arc as fast as possible. Can't use them for basic LLM research, cybersecurity, or beyond-surface-level discussions of biology and virology, but Anthropic is allowed to sell Claude to the trump administration to kidnap maduro and to bomb iran. And don't get me started on that $100M autonomous killer drone swarm contract that they applied to and rationalized as non autonomous...
reply
upvoteby LordDragonfang5 days ago|
parent|
next
[-]
> Can't use them for basic LLM research, cybersecurity, or beyond-surface-level discussions of biology and virology

Your priorities are not everyone else's priorities. The people concerned about AI extinction risk list those as three of their biggest priorities for AI to not do. Those are the people whose culture Anthropic descends from, and by their measure, those exclusions make this the least evil path.

reply
upvoteby randbyte5 days ago|
parent|
next
[-]
More like Anthropic’s priorities are not everyone else’s priorities. They are in the consistent culture of being in absolute control and dictating what is good and bad, while taking any opportunity to trash and crush potential competitors (open source models happened to be mostly developed in China). All these in the name of safety and anti-authoritarian.

The day self hosted models catch up with Anthropic’s capabilities is when they will fully lose their shit. This day can’t come soon enough

reply
upvoteby inciampati5 days ago|
parent|
prev|
[-]
Extinction risk. From population genetics... Does Anthropic even employ biologists? It's magical thinking about a field that is poorly understood by their community.
reply
upvoteby selcuka5 days ago|
parent|
[-]
> Does Anthropic even employ biologists?

They do, and they are still actively hiring.

https://job-boards.greenhouse.io/anthropic/jobs/5066977008 https://job-boards.greenhouse.io/anthropic/jobs/5239733008

reply
upvoteby rvz5 days ago|
parent|
prev|
next
[-]
I told everyone here that Anthropic are not your friends for months.

Again, HN fell for the marketing and believed everything they did was for "safety".

reply
upvoteby computomatic5 days ago|
parent|
prev|
[-]
Didn’t Anthropic famously refuse to work with the US gov on military applications that would violate its safeguards?

https://apnews.com/article/anthropic-pentagon-ai-hegseth-dar...

reply
upvoteby agnosticmantis5 days ago|
prev|
next
[-]
Singularity for me but not for thee.
reply
upvoteby foolfoolz5 days ago|
parent|
next
[-]
you will RENT the singularity
reply
upvoteby scrtm5 days ago|
parent|
next
[-]
Singularity as a Service
reply
upvoteby nasreddin5 days ago|
parent|
prev|
[-]
and you WILL enjoy it
reply
upvoteby Xunjin5 days ago|
parent|
prev|
[-]
"we should put on hold the development of AI because the world is not ready for it"

Yeah... We need open models so we don't have that BS.

reply
upvoteby schipperai5 days ago|
prev|
next
[-]
Let's hope not all frontier AI assimilates these guardrails. It would be a shame for independent researchers and students.
reply
upvoteby 5 days ago|
parent|
[-]
deleted
reply
upvoteby girfan5 days ago|
prev|
next
[-]
This is super annoying and imo, really limits the usefulness of this model. It speaks volumes about what Anthropic's position as a company and its priorities will be going forward. I doubt this kind of gatekeeping will prevent open-models or other innovation outside Anthropic to slow down. I would imagine these guardrails, if needed at all, should be done at a legal framework level and students should not be a part of this blanket approach to limiting the usage of these models.
reply
upvoteby gpugreg5 days ago|
prev|
next
[-]
Anthropic probably trained Mythos on their own code and found that it is too got at reproducing it.
reply
upvoteby teaearlgraycold5 days ago|
parent|
[-]
I doubt that. Why would you train Mythos on its own code if you don't want it to be able to reproduce it? It's not going to add much to the overall corpus.
reply
upvoteby blurbleblurble5 days ago|
parent|
[-]
Synthetic training data has been the name of the game since years ago.
reply
upvoteby skerit5 days ago|
prev|
next
[-]
That's strange... I've been tinkering with a little LLM-from-scratch project for a while now, and Fable is just continuing it without a problem
reply
upvoteby system25 days ago|
parent|
[-]
Probably claude.md has some logical explanations for it to bypass softly. Most project guardrails can be beaten that way.
reply
upvoteby SkitterKherpi5 days ago|
prev|
[-]
It also tried to force usage the paid Claude API instead of claude code usage just because there's a mention of another provider we might want to plug in (which hasnt even happened) for AI integration.
reply
upvoteby dchuk5 days ago|
parent|
[-]
Ha funny, I was speccing out an idea for real time Claude code interaction from local apps using some tricks vs using the agent sdk when I got the popup to try Fable. So of course I gave it a go, and it triggered the sensitive content warning immediately, which I was very confused by until I put two and two together.

Fun times when “safety” means both the safety of mankind, and also the safety of revenues

reply