It's both.
replyWe haven't had phones running laptop-grade CPUs/GPUs for that long, and that is a very real hardware feat. Likewise, nobody would've said running a 400b LLM on a low-end laptop was feasible, and that is very much a software triumph.
> We haven't had phones running laptop-grade CPUs/GPUs for that long
replyAgree to disagree, we've had laptop-grade smartphone hardware for longer than we've had LLMs.
Kind of.
replyWe've had solid CPUs for a while, but GPUs have lagged behind (and they're the ones that matter for this particular application). iPhones still lead by a comfortable margin on this front, but have historically been pretty limited on the IO front (only supported USB2 speeds until recently).
The iPhone 17 Pro launched 8 months ago with 50% more RAM and about double the inference performance of the previous iPhone Pro (also 10x prompt processing speed).
replydeleted
reply>triumph
replyIt’s been a lot of years, but all I can hear after reading that is … I’m making a note here, huge success
There’s no use crying over every mistake. You just keep on trying until you run out of cake.
replyIt's hard to overstate my satisfaction!
replyboth, tbh
reply