“You Could Have Invented Parser Combinators”
“You Could Have Invented Container Runtimes”
“You Could Have Invented Git”
Given the references to “mathematicians”, I think this reaction is more about an unfamiliarity with the concept of applied mathematics, which is ironic for practitioners in a field containing so much that is (or should be) regarded that way.
Software used to be all about “discrete math”, logic, maybe a bit of queuing theory…and now suddenly linear algebra and statistics became important. Don’t panic, it’s just another textbook on the shelf.
What this particular mathematical writeup seems to address is ‘how does someone who is already up to speed on the mathematical underpinnings of attention kernel neural networks get brought up to date on what they need to understand in order to see this new algorithm as obvious’
Which is a smaller audience to be sure, and - given how fast this field has grown - somewhat disingenuous in its implications.
This kind of title is appropriate for a piece aimed at helping someone who is a little behind the curve catch up. In the field of deep learning it should be obvious even to the people inside it riding that curve that it is moving really fast and that this kind of content only works for people who are running to keep up.
[1] https://medium.com/@gtrevorjay/you-could-have-invented-conta...