And at least with Moore's law, we had some understanding of the physical realities as transistors would get smaller and smaller, and reasonably predict when we'd start to hit limitations. With LLMs, we just have no idea. And that could be go either way.
Not from me you haven't!
> "they've hit a wall, no more data, running out of data, plateau this, saturated that"
Everyone thought Moore's Law was infallible too, right until they hit that bend. What hubris to think these AI models are different!
But you've probably been hearing that for 3 years too (though not from me).
> Models keep on getting better, at more broad tasks, and more useful by the month.
If you say so, I'll take your word for it.
Since then, all improvements came at a tradeoff, and there was a definite flattening of progress.
Idk, that sounds remarkably similar to these AI models to me.
I dunno. To me it doesn’t even look exponential any more. We are at most on the straight part of the incline.
So where is that?