i'm looking forward to Qwen3.8 27B launch to see how much models have peaked at a given size.
replyit might already be time to start burning the best small models onto hardware since it's possible they can't get much better at many tasks like knowledge recall due to the inherent information density limits for models at a given size.
The cartridge could be a small mac-mini type unit connected and powered over thunderbolt. If it included like an m5 or m7 with 64GB of memory and a PCIe5/6 4TB Nvme it would be amazeballs. Hopefully when the bubble corrects and hardware advances and prices reset something like that will become available.
replyJust even comparing compute from 10 years ago (Apple silicon vs Intel) and it's significant. 20 years it gets crazy. My first computer was an 8 bit 6502 with 64K RAM and a 128K floppy drive (I think, it's fuzzy). Everything amazing now will look quaint in due time.
It is not linear anymore, take in consideration the Moore's law, the curve is nearly saturated now and gains in performance and memroy are not accelerating any more,
BUT there is some hope with new different technologies, like the PHOTONIC chips , doing GEMMs through light particles instead of electrons
reply> Imagine a cartridge with a physical model on it.
replyI can finally have my own Dixie flatline. Cool.
> Dixie Flatline
replyIn case some did not know: also the movie (actually TV series) is finally happening.
# Neuromancer - Official Teaser ( https://news.ycombinator.com/item?id=49055037 )
very interesting idea. i didnt think of that. i was just assuming youd have an additional one of these in your phone for actual lightning fast local inference
reply