There will be multiple notations (MetaMath, Lean, and essentially Frege's notation everyone learns in high school), and we could try to identify how the neural networks represent them as vectors (or vector combinations). The moment formal logic can be connected to the reasoning representations, regularization can be reduced to eliminating internal inconsistencies.
Even the original transformer architecture makes this clear. It had an explicit "encoder" phase and then a "decoder" phase. Modern LLMs collapse the two together, or are sometimes described rather confusingly as being decoder only. But what they're doing is more or less the same.
Yeah, the encoder and decoder stuff is explicit, but the internal structure in generated during training. I don't think the big labs were doing this back when I did the research; no one was back in '24.
I just didn't get round to publishing for years, because I have a day job.
By the way, it still works! I tested it earlier this year on Qen3.6 and you still see improvements, so either a) no one actually paid attention, or b) it has more room to scale.
My impression from reading the literature is that there are a gazillion interesting ideas and findings published that nobody is picking up in production models. The big labs are researcher constrained, there just aren't enough hours in the day to keep up with the literature and integrate all the interesting ideas found there. So it's not surprising that your trick still works. It'd be even less surprising to discover nobody at these labs has read your blogs, or they have but never found time to experiment with them. Or, they tried, but there is no set of loops that improves some metrics without harming others - I would expect neural circuits to be misaligned across the middle layers so looping layers for one task would put a fault line in circuits for other tasks.
Then they have to trade off the extra GPU capacity needed to do the extra layers, and so on.