LLMs are hungry for tokens. Every day I see more startups claiming token usage at 50-100k per month. You can always brute-force your way in - just see HuggingFace's recent attacks.
replyIf throwing more money at inference while accumulating compounding technical debt is the new norm, then we are not solving the problem, and the solution space is already covered.
Perhaps there are marginal gains at the expense of quadrillion-params LLM models with 10x the cost and energy. We are simply making inefficiency more expensive, camouflaged by VC money and great marketing.
If that is not plateauing, then I guess I will have to reconsider what plateauing means.
> You can always brute-force your way in
replyThis is such unbelievable revisionism! Can you imagine in 2022 saying "Oh of course you can brute force your way to AGI if you spend enough money per month". Nobody thought that! Come on!