Ryzen AI are competitive with Macs in absolute and power efficiency terms. There is a Framework 13 with them I believe.
replyHumans own inference power efficiency by a much bigger margin
replyOwning a human is much more problematic, and anyway a human’s peak efficiency might be higher, but humans have to sleep and take breaks so overall probably not.
replyHumans also tend to hallucinate a lot, but it’s not polite to use that word for them. With humans we say they were wrong, because offending a human reduces performance and you can’t reset their contexts.
replyI just have a desktop at home that I run inference off of. It is a great setup and I don't find myself wanting to inference models directly on my laptop.
replyThat’s what I would do too, but I haven’t found a desktop build that can rival a Mac Mini or Mac Studio on performance per watt. I haven’t looked super hard, but it seems like Mac is in a different ballpark.
replyI mean, once it's a desktop, watts are pretty cheap, so it's a bit strange to optimize on that factor for the desktop form factor. For laptops it makes a ton of sense.
reply