upvote

points

by Gareth3211 hours ago |
comments
upvoteby alexpotato1 hours ago|
[-]
Had the same reaction so had Grok create a script to:

- find a free GPU droplet on digital ocean

- fire it up

- pull in a snapshot of the model + extra files/packages etc

- set up a ssh tunnel so that the localhost:8000 routes to the above

Then I just configured OpenCode to use the above and was off to the races.

Works out to be about ~$2/hr all said and done which isn't bad as I only pay when I'm using it (but could get expensive with 24/7 running)

reply
upvoteby embedding-shape32 minutes ago|
parent|
next
[-]
> Works out to be about ~$2/hr all said

What GPU you end up with for that price? Vast.ai (https://cloud.vast.ai/?priceInstanceHourlyMax=2) has a bunch of setups available to reach 192GB VRAM under $2 :) Quick skim showed 4x48, 2x96 and 8x24, all for under 2 buckaroos or around there.

reply
upvoteby b11248 minutes ago|
parent|
prev|
[-]
Awesome, I used Claude to write a small python script to do the same with Linode's API. The only difference is I setup a persistent drive, and with Linode you can boot off of it. So my biggest start up lag is ~ 2 minutes to deploy + boot, then maybe 2 more to warm the model.

I actually dislike LLMs. But I'm a realist, and on-demand compute like this is massive cost saving measure.

(persistent drives are relatively cheap, compared to a box with several GPUs.. or even one. I find it worth the expense)

reply