The https://chatjimmy.ai demo was impressive.
Once models settle down this makes sense. Imagine a cartridge with a physical model on it. You purchase a cartridge and stick it in your computer/phone/server. Want to upgrade? By a new 'cartridge'.
This should bring inference cost down dramatically, I wonder how OpenAI/Anthropic feel about that.
it might already be time to start burning the best small models onto hardware since it's possible they can't get much better at many tasks like knowledge recall due to the inherent information density limits for models at a given size.
Just even comparing compute from 10 years ago (Apple silicon vs Intel) and it's significant. 20 years it gets crazy. My first computer was an 8 bit 6502 with 64K RAM and a 128K floppy drive (I think, it's fuzzy). Everything amazing now will look quaint in due time.
I can finally have my own Dixie flatline. Cool.
In case some did not know: also the movie (actually TV series) is finally happening.
# Neuromancer - Official Teaser ( https://news.ycombinator.com/item?id=49055037 )
It’s composed of 4-bit multiplier cells that compute all 16 possible results in parallel. The top metal wiring layer physically selects the one that corresponds to a multiplication with that cell’s constant weight, and routes it to the next layer.
They won't sell/rent/license the weights to an end user at any price because they don't trust your security.
Now your robot can respond sarcastically when you ask for chicken nuggets. Again. It also doesn't dent your walls anymore.
Imagine if like instead of having a specific Mac Plus ROM, you had a thing that looks like a fat ASIC that can hold models sitting on a slotted daughtercard directly next to the CPU and RAM.
Fable is nice, but still requires a lot of guidance for large scope tasks.
But yeah, for things like programming, if it can do linux and python and some go and sql and javascript, larger domains can be threaded with LORA