To put it more simply, people with academic credentials should not demand acknowledgement of their current intellectual work while denigrating and ridiculing the importance of very similar work done in the past.
And that's where Schmidhuber goes off the rails: publicly shaming published papers into citing you isn't good academic practice. It's bullying.
You can't claim independence from past work simply because you didn't look directly at it. The job of an academic researcher is to know the landscape of relevant ideas, where they come from, where they're going, and to hopefully contribute a few new good ones.
Citation chains should extend back from your work, along a reasonable line conceptual inheritance, back to a reasonable point of origin. Schmidhuber has different definitions for both of these reasonables than the bulk of the ML research community, to a point that makes him difficult to satisfy.
For example, take a look at Albert Einstein's Google Scholar profile. He's not the top cited physicist. Not even close. It's because other researchers don't explicitly cite his papers. https://scholar.google.com/citations?user=qc6CJjYAAAAJ&hl=en...
Same with Tim Berners-Lee and the World Wide Web. Imagine if his original paper were cited every time someone deployed a web site.
If I’m in the private sector, and I rediscover something from first principles, it is not my responsibility to go search all academia to see if someone’s done it before so I can cite their work.
If I rely on a code library that doesn’t explicitly cite papers it was built on, it is also not my responsibility to go find all the papers that it might’ve been built from and cite those papers.
Spamming citations is unnecessary.
Eh, I think the correct answer is: read it, then cite it.
You're not really supposed to cite something without reading it, as it might say something different than you think. But sure, citing it w/o reading it is better than not citing it at all.
But if you build on them you should have read them. I don't know about the specifics and I don't know if Schmidhuber is out of line or not, and citations and impact factors are a terrible mess, but generally speaking, you are responsible for finding and reading and citing any related work that needs to be cited, and if you work on neural networks in an academic context you probably have been forced to read that particular one at some point. Citation obligations don't just disappear because you don't want to do the research.