undefined

points

[-]

I tried it on my M1 MacBook Pro. It's slow but surprisingly smart as a general purpose LLM. Maybe GPT-5.3 level. I gave it a bunch of tools and it can search the internet, make product recommendations, document, code, etc.

by alexpotato1 hours ago|

parent|

[-]

Had the same reaction so had Grok create a script to:

- find a free GPU droplet on digital ocean

- fire it up

- pull in a snapshot of the model + extra files/packages etc

- set up a ssh tunnel so that the localhost:8000 routes to the above

Then I just configured OpenCode to use the above and was off to the races.

Works out to be about ~$2/hr all said and done which isn't bad as I only pay when I'm using it (but could get expensive with 24/7 running)

by embedding-shape31 minutes ago|

parent|

[-]

> Works out to be about ~$2/hr all said

What GPU you end up with for that price? Vast.ai (https://cloud.vast.ai/?priceInstanceHourlyMax=2) has a bunch of setups available to reach 192GB VRAM under $2 :) Quick skim showed 4x48, 2x96 and 8x24, all for under 2 buckaroos or around there.

by b11247 minutes ago|

parent|

prev|

[-]

Awesome, I used Claude to write a small python script to do the same with Linode's API. The only difference is I setup a persistent drive, and with Linode you can boot off of it. So my biggest start up lag is ~ 2 minutes to deploy + boot, then maybe 2 more to warm the model.

I actually dislike LLMs. But I'm a realist, and on-demand compute like this is massive cost saving measure.

(persistent drives are relatively cheap, compared to a box with several GPUs.. or even one. I find it worth the expense)

by chorlton20802 hours ago|

prev|

[-]

Does it need to respond fast? For important applications, I'm sure we'd all be fine waiting 20 minutes for a high quality, usable answer. Or is it the need for interative refinements that make speed relevant?

by jonplackett1 hours ago|

parent|

[-]

It requires patience but it’s more like waiting 5 mins for it to do tasks. You need to be much more involved though and do things slower than Claude where you can trust it to do a lot of tasks at once. It doesn’t have the context for that

by dominotw52 minutes ago|

parent|

prev|

[-]

if you are so sure about what the final shape of your output is then its prbly not a common use of ai