I have no numbers to back up my argument, but smart phones are very power efficient by their nature are they not? I can play a 3d game, with impressive graphics, on a tiny device powered by a battery... With very little heat generated.
If your goal is to run LLM inference on a gpu in a power efficient manner, I bet a smart phone is a good place to start.
But yeah, these are great questions which are not obvious at all and should be answered when proposing such a system.