upvote
Here's a widely used toolkit: https://github.com/p-e-w/heretic

Distributing the vectors themselves isn't (yet) common practice, because people have gotten used to just putting the full modified weights up on HuggingFace's huge free storage.

reply
I am, in fact, I’m offering something much better, a compact format that exactly restores an abliterated model from a tiny JSON file:

https://www.reddit.com/r/LocalLLaMA/s/iksvmXBzuC

reply