What makes it work is that it reads its own hidden states and weight matrices -> then writes a few-line Python script to fix the weights directly (within a 20-forward-pass budget, so it can't just brute-force guesses).
After skimming through the paper and comparing it with how SEAL works (where the model still relies on RL and gradient descent to apply its self-edits), I understood it something like that :)
But I think there are comments in the thread that explain the main concept far better than I do.
I think the real question is going to be whether this is more efficient than just conventionally training the model.
In this case, they just ask the model what the weights ought to be.