Show HN: Distilling DeepSeek into GPT-OSS doesn't transfer censorship. Try it
(www.ctgt.ai)
I know not all models can be easily abliterated or uncensored, but is there a reason to start with a model that is still censored?
replyI’m thinking this makes fullt sense because distillation is only additive, not subtractive. So it does not remove knowledge (if we can define censorship as removal of knowledge).
replyMost censorship isn't "removal of knowledge" but "installation of behavior that prevents some knowledge from being revealed or applied in certain ways".
replyThis behavior can, in turn, be transferred via distillation. But, evidently, financial domain wasn't entangled enough with the censorship behaviors for them to bleed through, in this case.
Consider that LLMs are trained on the corpus of the internet, and (simplifying) consequently give the average answer of the internet. If the desired answer of the censorer is contradictory to this, then it requires additional training data to get the model to act a certain way.
replyFYI the scrolling on iPad with trackpad is broken. A full swipe on the trackpad is about 1 inch of screen movement.
replyScrolling on desktop is also broken.
replyThanks for letting us know. Is this is on the research post?
replyDeepseek is, with difference, the most "Western" of Chinese models, so it's a bit perplexing that it was chosen to test this hypothesis.
replyI didn't run any benchmarks but I played around a little, and after getting around the API-level filter Deepseek V4's answers about "China-sensitive content" aren't any different from what I get from Claude and ChatGPT.
You can see exactly what prompts we used and the results here: https://github.com/CTGT-Inc/lineage-eval/tree/main/data
replyWe found V4 Flash was significantly more censored than the baseline.
Could just be resources available? Deepseek is the easiest to get up and running on hardware that's pretty readily available:
reply unsloth/DeepSeek-V4-Flash-GGUF 4bit ~140GB
unsloth/Kimi-K3-GGUF 4bit ~1.5TB
unsloth/GLM-5.2-GGUF 4bit ~400GBSo, there is no subliminal learning in this situation, under what conditions would we expect it. I find a transfer attack to be a bit far fetched but it’s definitely interesting.
replyIf we trained from random initialisations on DeepSeek output (that didn’t explicitly contain the political questions) we would expect transfer? And if we fine tuned a model pretrained elsewhere on Deepseek output?
What is the line?
It's most likely to occur when distilling a Chinese model from a Chinese base. We plan to do compliance geometry analysis in the future to see what is structurally changing in the model when distillation causes it to start refusing or whitewashing.
replyIt'd be interesting to use this technique to create a running tally across all models of which models are censored on what topics
replyHijacking scroll behaviour in 2026 is wild.
replyso interesting!!
reply