That’s pretty fascinating, Apple has some innocuous LLMs and transformers baked into its devices and leveraging their neural chipset
replySo I could see something like this where the neural chipset has an LLM that cant be so easily updated baked into it, until you get a new device
Exactly, it'd be the same as regular chip designed evolving. You get a specific model version baked into the chip, if it does what you need then it's fine. If you need more capability in the future, you just buy a new chip.
replyI also think the dynamic would be really different if model inference can run at ridiculous speeds. You could make a genetic algorithm loop around it, so it can generate a population of proposals at each step, then have those tested and whittled down iteratively. If inference happens at thousands of tokens per second, then from user perspective it would still be really fast, and even a small model could solve complex problems.