Tried with S25+ (SD 8 elite). 0.82tok/s(4B L model). It's barely useful speed but it's pretty impressive either.
replyIt depends on your idea of decent speeds and what you would use it for. I just tried it on a laptop with an AMD HX 370 running on battery in power save mode and it's not especially impressive, although it runs much better in balanced or performance mode. I gave it the prompt "write a fizzbuzz program in rust" and it took almost a minute and a half. I expect it to be pretty terrible on an SBC. Your best bet is to try it out on the oldest hardware you have and figure out if you can tolerate worse performance.
replygood idea, will test that out
replyRK3588 uses a 7 year old CPU design and OrangePi 5 looks expensive (well over $100).
replyA used sub-$100 x86 box is going to be much better
you're right. For my purposes, I was thinking of something I could use if I wanted to manufacture a new (smallish) product
replyI'm going to attempt to get it running on the BeagleY-AI https://www.beagleboard.org/boards/beagley-ai
replySimilar form factor to raspberry pi but with 4 TOPS of performance and enough RAM.