It's definitely an exciting time, but in terms of advancements in the state of the art, there is a lot of low-hanging fruit left to pick. There IS a bottom, however, as you can only encode so much "knowledge" in a small number of parameters.

This feels to me a lot like what the early days of what radio or aviation must have been like. Or, heck, microcomputers even.

Today, data systems and algorithms can be deployed at unprecedented scale and speed. Unintended consequences will affect people with that same scale and speed

—Michael Chapman

My favourite conspiracy theory lately is that the above isn't a silly fairy tale, that we actually used to live much much longer -- until the common cold came on the scene, and the sequelae dramatically shortened our lifespans. Today we dismiss it as "just a cold" unbeknownst of what it robbed us from.

Only after the current generation(s) of doctor(s) dies. And only if you make this in pill-form. Otherwise people will be people and won't even go to the gym.

It might be also the reverse, they develop a powerful+personalized drug that brings heaven on earth to your neurons (first time heroin experience + sexual gratification + childhood fulfillment + extremely addicting etc etc etc).

-----

Now that I think of it I'm gonna go with the latter.

Large models still are quite far ahead, don't be fooled that even Gemma:31b (which is better than the 12b overall) is anywhere close to big models.

There is definitely room for optimization, but fundamentally, for complex tasks, you need visible small gradients for accuracy that allow the model to be trained on (and consequently be followed during inference). For example, if you specify in instructions not to write code but ask coding question, Gemma will still write code. Whereas Gemini/Claude will pick up on that and follow your instructions better.