undefined

points

[-]

I don't know much about chess engines; do they still use hand-tuned algorithms, or are they more like AlphaZero, where they learn through self-play to beat any/all possible human contenders? I don't believe DeepBlue was automated to that extent, but it may have been.

In the latter case, the chess example would tend to support the Bitter Lesson, rather than refute it.

I would also be VERY slow to claim that general-purpose models will never be competitive at chess. It wasn't so long ago that transformers couldn't add two-digit numbers reliably without resorting to tool use. They are now as good at "mental arithmetic" as any human savant. It wouldn't surprise me at all to see someone come up with a model that just happens to be really, really good at leveraging the portions of its general training data having to do with chess.

In fact you could argue that AGI demands such a model, if we are to assume that LLMs are a guidepost in that direction.

by inigyou2 hours ago|

parent|

[-]

AFAIK Stockfish still remains a traditional (some-variant-of-) minimax-search engine, but now uses a microscopic neural network to evaluate the positions at the maximum search depth rather than heuristics. I expect any pure ML engine can learn to exploit any fixed heuristics, but now Stockfish is trying to balance both strategies, which still heavily ends up on the side of raw search depth.

by dmoy7 hours ago|

parent|

prev|

[-]

I don't know anything about the last 8 years of chess engines, but yea maybe 8-10 years ago AlphaZero shit all over e.g. stockfish.

by brucehoult3 hours ago|

parent|

[-]

I guess you missed Leela then.

And in 2020 Stockfish 12 adding some NN evaluation. And then in 2023 Stockfish 16 entirely removing the classical position evaluation code.

https://stockfishchess.org/blog/2023/stockfish-16/

by applfanboysbgon6 hours ago|

parent|

prev|

[-]

DeepBlue beat Kasparov with essentially raw compute thrown at alpha-beta search. That does support the premise of the bitter lesson in general. But that does not mean the bitter lesson is correctly being applied here. The point is that even if throwing raw compute at a task is better than careful human-crafted algorithms, it's still task-dependent. The current trend with the people blowing hundreds of billions of dollars is developing an all-purpose model that is everything to everyone, but you don't need hundreds of billions of dollars to create a task-specific model that outperforms their model at a given task.

> I would also be VERY slow to claim that general-purpose models will never be competitive at chess.

This is not the claim. The claim is that for the same amount of compute, a general-purpose language model will never beat a Chess model. I'm dubious, but allow for the possibility that a language model could eventually compete at a top level against humans with enough compute. However, it will never compete with a dedicated Chess model with similar resources. Training a model for a specific task with the same amount of compute will outperform training a general-purpose model with the same amount of compute. This should be common sense, right? The bitter lesson was only about compute over human algorithms, not at throwing compute at a generalised domain over throwing compute at a specific domain.

You made arguments against two claims that I did not make (that I was trying to refute the bitter lesson or that I claimed that LLMs could never be competitive against humans at Chess), so I'd like to ask you read my statements a little more carefully this time.