upvote
Citing 2023 makes it seem like this is newer than it is. Compression, prediction and intelligence have long been known to be deeply connected.
reply
You expect every blog post to find the earliest relevant paper to cite, just so one could look at the year (without reading said paper - which would have made clear that the connection isn’t recent) to assess novelty? I don’t think that’s reasonable.

It’s a blog post. If it was, say, a peer reviewed paper by Hinton or LeCunn that fails to cite Schmidhuber, that would be reasonable criticism in my opinion. (Spoiler: they fail to cite him)

reply
Why would blog posts not be subject to such criticism?

Either the author knew of prior work that argues the same thing and they ignored it, or they didn't know. And if one writes a 1000+ word article premised on this idea, wouldn't one be presumed to know at least in which century the idea originated from?

Arguably these kind of blog posts should be more subject to such criticisms, because the blog posts purport to "teach" the general public about a concept in an authoritative tone (or at least the author seems to pose as knowledgeable in the subject), while for academic papers, everyone who actually reads the paper knows where the ideas came from anyway and it's mainly an issue of attribution (and maybe about fairly distributing the citation count...)

reply
> Why would blog posts not be subject to such criticism?

You're asking why casual comments from amateurs made for fun on the internet shouldn't be held to the same standard as those made by funded career academic experts writing for other experts over months and meant as part of the permanent record of the field?

Personally, I think that's a bit like asking why a friend having you over for dinner isn't always an elegant 7-course meal with wine pairings. I guess you can expect that if you want, but to me it sounds like a child expecting to go to Disney every day: ignoring the economic realities of the situation is a recipe for eternal disappointment.

reply
I think you shouldn't criticize the single blog post in vacuum.

If the blog is a personal collection of "today I've learned" then different standards apply then "my blog is my research publishing platform".

Both are valid blogs. The fact that it ended on HN front page didn't mean it was targeted towards this audience

reply
> You expect every blog post to find the earliest relevant paper to cite

This should be expected out of everyone. If you don't respect the reader enough to do this, why should we read your posts?

I think papers should be retracted for not citing prior art, even if you weren't aware of it.

reply
If the blog were about calculus, and stated that an elegant proof of the Fundamental Theorem of Calculus could be found in such and such undergraduate textbook, would you be upset that the citation wasn't to either Newton's or Leibniz' work?
reply
reply
The two contributioms that come to mind are the Hutter cash prize for the best compressor of the English Wikipedia and the work on PPM compression-based text classification by the late Prof. Ian Witten's group at Waikato (NZ) [1,2]. The model that best compresses the input string was likely generated by the distribution from which the compression model was 'trained'

[1] https://news.ycombinator.com/item?id=27440093

[2] https://www.computer.org/csdl/proceedings-article/dcc/2000/0...

reply
That compression and prediction are deeply connected is obvious from simply considering how compression algorithms work.

Nothing about the concept of "intelligence" is nearly as well defined as the other two, nor as well agreed upon.

reply
"Understanding" is clearly linked to compression. Taking observations and coming up with a more compact representation that explains them, analogous to coming up with a compact set of axioms that generate facts, or a small Turing machine or short program that generates a list of strings.

Intelligence is a broader concept but definitely involves understanding how a system/envoronment works and making predictions about its unfolding, especially actionable ones that allow you to steer that state towards some goal states.

These are not new.

reply