Well, everything—even human life on Earth—is just temporary, but RAM supply lagging centralized-AI-driven demand increases continuing to squeeze the consumer market may not be a short term phenomenon.
> LLMs seems quite simple, maybe we'll be able to print/assemble at home our own chips with the desired models/weights.
So, the solution to the RAM crunch is “everyone has their own home chip fab and deals with the raw material supply and hazardous waste disposal”?
I...don't imagine so.
My idea was more like you get some pre-made chips, that you can maybe assemble together configure at home with your desired models.
Maybe each one of this chip is a layer, so you can stack as many layers as you want.
This has a lot to do with how MCTS works BTW. The current best path is often only the current best path because a lot of investment has been sunk into it. If you were to put equal resources into a different path you may find that it was actually far better. It is just that the early rollouts favored the other 'best path' so you sunk a lot of resources into that one. We are very early in our exploration of LLM architecture. I highly doubt we are anywhere near the best path right now.
The diffusion models are interesting, but those also seem hacky.
I think the next form of AIs will be simpler and more abstract.
The building blocks of our brain don't have the notion of a "token" embed into them, it's lower level that that.
I think first step is to find a better way to represent information.
LLMs shouldn't "compute" stuff using language tokens, but some other, more efficient logical mechanisms. LLMs should first "feel" the solution, reason internally in that optimised space, then, only when interacting with a human should it convert all that into actual tokens/language.
The other option is maybe hook up humans to EEG or the likes and map their brains while they solve different kinds of problem, or just see and feel the world around them