undefined

points

[-]

One of the advantages of LLM's is that you can set up a task list and tell it to burn through those tasks overnight. It's a rare night that Claude isn't busy for me, and I do burn through my 20x subscription, sufficiently that I downgrade from fable around about ... now in the week...

by hellohello212 hours ago|

prev|

[-]

"You most likely can't utilize the workstation 24/7. Your work hours will be concentrated into 6-10 hours per day."

I have agents running 24/7 doing research, in fact I would argue this how they will be used for most programming tasks in the near future. For chatting, I agree local inference makes no sense. But for tasks that run continually, I'm not so sure. Personal computers took a while, local inference will too, but I think it will happen.

by manmal9 hours ago|

parent|

[-]

Sorry for the snark, but are you trying to cure cancer? What could possibly need 24/7 research in our domain, that doesn’t need your input every 30 minutes?

by hellohello29 hours ago|

parent|

[-]

Normal boring CS scientific work. Just running running my experiments, reproducing other papers, etc. A lot of it does involve the agent waiting for some computation, but the fact that it resumes independently when I'm sleeping is kind of the point (+ usually I have several running in parallel).

I'm not trying to cure cancer, although I do hope people who are use LLMs. ;)

by manmal9 hours ago|

parent|

[-]

I didn’t expect an answer I can actually agree with - keep up the good work ;)

by timfsu13 hours ago|

prev|

[-]

Subscriptions are, and will likely remain, the best deal in town. Unfortunately, larger companies aren't able to do that. When your monthly token costs are in the $5-10k range, the local inference starts to look a lot more attractive

by swat5358 hours ago|

parent|

[-]

Don't subscriptions have limits and resets? Making them not very usable for an ongoing operation?

by ltbarcly312 hours ago|

parent|

prev|

[-]

In the case where you pay for tokens without a subscription, the analysis is still very much not in favor of buying hardware.

The assumption previously used was that you can run a Sol level model on an M6 or whatever hardware $20k gives you. That is not true, it was an assumption made to show that even giving your own hardware every reasonable advantage it still loses.

Lets compare buying tokens of the best model you might run on your own hardware (still being unrealistic in favor of your own hardware) vs that same class of model on the market. I think one of the best models you might be able to run is GLM 5.4, but lets just look at chinese models generally:

$20k workstation, best case: $15k M5 Ultra 512GB, 36-month amortization, ~$440/mo. Runs a GLM-5.3-class model at ~30 tok/s. Saturated 24/7 it produces roughly 58M output tokens/month.

Buying those tokens:

  DeepSeek V4 Pro  @ $0.87/M   $50
  Kimi K2.6        @ $4.00/M   $232
  GLM-5.3          @ $4.40/M   $255
  Kimi K3          @ $15.00/M  $870  (does not fit on the box)

The economics can never work in your favor for buying your own hardware here, unless you can utilize it or sell excess capacity and you have access to nearly free electricity. The reason is someone else can buy the same hardware at scale (or realistically more efficient hardware), park it somewhere with very cheap electricity, and sell tokens. They can get very high utilization that you are not likely to get.

And keep in mind I am giving 'your own hardware' no overhead or maintenance cost, despite your condition that it's in a large corporate environment. In reality corporate IT would make it almost impossible to set up and your would need huge lead times to buy the hardware and get it installed.

by maherbeg11 hours ago|

parent|

[-]

Thank you for being explicit with the math!

So yes, at that speed for sure. But if the speed goes up? or the ability to batch at the same speed goes up? The economics start to shift. The gap is much closer, and you'd end up with a box you can still use or sell later.

Subscription pricing is still the best though!

by ltbarcly38 hours ago|

parent|

[-]

As speed goes up the cost / Mtoken will necessarily go down at roughly the same ratio so it will wash out. The still use hardware or sell hardware value is factored in to the amortized monthly cost, it assumes a 3 year markdown, and does not factor in the cost of money which should almost cancel the resale value in the end, which I think is quite accurate (any residual cost on a graphics card after 3 years is so small compared to the current price it should be discounted and in included there).

Where you might win by owning your own hardware: - Hardware costs go up, and thus api costs go up. You've locked in your pricing. - Chinese/Open models become illegal/hard to access the way we do now. OpenAI and Anthropic are trying very hard to build a regulatory capture scheme to do this. I think they will be unsuccessful because China just won't participate.

by mrkstu9 hours ago|

parent|

prev|

[-]

Leaving out that apparently high RAM Mac's apparently no longer lose value over time...

by ltbarcly38 hours ago|

parent|

[-]

There is a critical shortage right now, if the AI datacenter boom slows the prices could crash very quickly.

by slashdave11 hours ago|

prev|

[-]

Some people neglect to factor in electricity costs. Some homes have very expensive service.

by mathisfun12312 hours ago|

prev|

[-]

> - You most likely can't utilize the workstation 24/7. Your work hours will be concentrated into 6-10 hours per day.

isn't the whole point of all this ..... agents? isn't that what literally everyone is always clammering about in these threads? in which case the workstation is useful 720 hours out of 720 hours.