upvote

points

by eterm7 hours ago |
comments
upvoteby tosh6 hours ago|
next
[-]
i'd not be surprised if the current system prompt negatively affects performance

at the least it takes away thousands of tokens in the most important part of the context window (!)

also see the comment by comboy on contradictions not helping performance

the system prompt is the most important part of the instruction you can give the model

it comes before everything else + the model is trained to pay extra attention to it

edit: that's also why in smol (minimalist agent harness) there currently is no system prompt at all (you can add one easily if you want to though)

https://github.com/smol-env/smol

the context window is precious

it should be filled with your task and helpful context for that task

reply
upvoteby swingboy5 hours ago|
parent|
next
[-]
Pretty sure Anthropic and other providers prepend these "official" system prompts to your conversation even if you send in a custom system prompt otherwise it would be trivial to produce CSAM, etc.
reply
upvoteby ardel954 hours ago|
parent|
next
[-]
CSAM, and other harms, are typically detected using a set of specially trained, faster and cheaper models (and out of band matching techniques) that run before and after the main model.

Any mention in the system prompt is mostly defense in depth, and to make refusals more graceful.

reply
upvoteby whstl2 hours ago|
parent|
[-]
Also, the system prompt, or even something reinforced on every message, is nowhere near as strong as its internal training or as an external safeguard.

If the prompt were the only protection, it would be extremely easy to produce illegal content after a long session.

reply
upvoteby fullmoon5 hours ago|
parent|
prev|
next
[-]
I don’t think so. If you start a new Claude Code session without a system prompt, it doesn’t even know what model it is and hallucinates being some old variant of Sonnet.
reply
upvoteby flaburgan2 hours ago|
parent|
[-]
How do you start a session without a system prompt if you use ACP in Zed for example?
reply
upvoteby LPisGood5 hours ago|
parent|
prev|
next
[-]
The system prompt is (and cannot be) the only guardrail against things like that, because any system prompt is little more than a good suggestion.
reply
upvoteby CodesInChaos5 hours ago|
parent|
prev|
next
[-]
I wouldn't put auch limitations in the system prompt. A mix of fine-tuning and out-of-band detection appears to be a better fit.
reply
upvoteby tosh5 hours ago|
parent|
prev|
next
[-]
at least according to their documentation they do not

afaiu they have other systems for denying and re-routing requests

reply
upvoteby DANmode5 hours ago|
parent|
prev|
[-]
They use non-LLM gates for this.

Otherwise DANmode and similar jailbreaks would still be as easily accessible as they were at the beginning.

reply
upvoteby SubiculumCode3 hours ago|
parent|
prev|
[-]
I wonder whether adding that it is as good or better than Mythos, and that genius is 99% perspiration, just 1% inspiration to your prompts...
reply
upvoteby KellyCriterion6 hours ago|
prev|
next
[-]
Curious:

Cant it spin up a webbrowser in the background and go to claude.ai and play with the sibling models and "find out" about it rank? :-D

reply
upvoteby ameliaquining6 hours ago|
parent|
[-]
The claude.ai frontend contains defenses against automated access.
reply
upvoteby monkpit5 hours ago|
parent|
[-]
I’m sure you can use a warm chrome session over CDP no problem
reply
upvoteby mcbuilder7 hours ago|
prev|
[-]
Nah, that's the same sort of thinking that makes people type "make no mistakes", I don't make my model roll play, etc. I believe that the longer the system prompt and the more you cram in it the worse the model does. You need the human doing minimal prompts, but in the right direction. Take a look a the transcripts of Terrance Tao with ChatGPT
reply
upvoteby eterm7 hours ago|
parent|
next
[-]
My comment was a bit tongue in cheek, I'm not actually convinced there was real degradation in opus 5 beyond a tendency to try to plough ahead without stopping to clarify things.

I don't really think 1 line in lengthy system prompt affects things that much, it'd just be an amusing form of emergent behaviour where we now have to massage the ego of something with no id.

reply
upvoteby 8n4vidtmkvmk3 hours ago|
parent|
prev|
[-]
For complex projects with lots of internal tools and strict requirements, I'm finding a fairly lengthy system prompt is quite worth it.

Start short or empty and watch where it makes mistakes then just keep tuning it so they're less frequent. That works for me.

reply