I think we use different units. In my system there are 3600 seconds per hour, and watts measure power.
replyOP probably means watt-hours.
replyAnd 0.5 tokens/s should work out to 1800 tokens at the end of the hour. Not 3600 as stated.
replySomething to consider is that input tokens have a cost too. They are typically processed much faster than output tokens. If you have long conversations then input tokens will end up being a significant part of the cost.
replyIt probably won't matter much here though.
Open router is highly subsidized. This might be cheaper in the long run once these companies shift to taking profits
replyBut why not cross that bridge then. By that time you might have much more optimized local infrastructure. Although I do see that someone suffering through the local slowness now is what drives the development of these local options.
reply> Cost wise it does not seem very effective.
replyWhy is this so damn important? Isn't it more important to end up with the best result?
I (in Norway) use a homelab with Ollama to generate a report every morning. It's slow, but it runs between 5-6 am, energy prices are at a low, and it doesn't matter if it takes 5 or 50 minutes.
> Why is this so damn important? Isn't it more important to end up with the best result?
replyYou’re wondering why someone would prefer to get the same or better result in less time for less money?