upvote

points

by Fr0styMatt8819 hours ago |
comments
upvoteby matheusmoreira18 hours ago|
next
[-]
Yeah. There's no way to verify what these providers are doing. The real future is running these models at home. Opus level inference on our own hardware would be a dream come true.
reply
upvoteby baq7 hours ago|
parent|
next
[-]
I dream of having an LLM in a box over usb bought off AliExpress for a year and change now.

The LLM in a box is something you can buy today, but it 1. doesn’t serve over usb by default 2. costs $100k for hardware (not counting electricity) at 100 tps 3. can’t buy this from AliExpress.

Better to put that $100k in t-bills and just buy tokens even at api prices.

reply
upvoteby rescbr3 hours ago|
parent|
[-]
I understand your point (and definitely want the same), but I do have an almost-AliExpress-LLM-in-a-box: it's an Thunderbolt eGPU dock (that I got from AliE, and it is USB-C...) with a RTX 4060 Ti with 16 GB of VRAM (bought locally for gaming before the price boom)

It's been awesome for embeddings and document OCR!

3D printing a case for it is on my todo list.

reply
upvoteby IncreasePosts18 hours ago|
parent|
prev|
[-]
How will anyone running home instances be able to compete against people paying some money running much more powerful models on much more powerful hardware?
reply
upvoteby Fr0styMatt8817 hours ago|
parent|
next
[-]
It’ll be interesting.

I’m using Qwen3.6:27B at home and mostly Sonnet/Opus (depending on the complexity of the task) at work.

You have to break things down into smaller chunks for the local models. For the bigger cloud ones they can do a lot of the broader thinking.

reply
upvoteby fragmede10 hours ago|
parent|
[-]
Time is money, but apparently now thinking is money as well. How much is it going to cost to think harder? If it's, say, $10 to use a bigger cloud model, it becomes easier to qualify the cost of thinking.
reply
upvoteby Bombthecat8 hours ago|
parent|
prev|
next
[-]
Yeah. There always will be a gab. And it will keep growing for the next years...
reply
upvoteby jimbokun15 hours ago|
parent|
prev|
[-]
At some point it will be hard for us to tell the difference.
reply
upvoteby RazorBucksICO6 hours ago|
prev|
[-]
The appropriate price is what the output is worth to you. Some people could pay $10,000/month, some $5 and feel like they were breaking even. There is a big jump between convenience and curiosity uses versus business critical.

OpenAI already charges enterprise users a premium purely for that title over on-demand, no-contract usage. Retail users get a good deal. People make a lot of hay about subsidies but this is a very sane approach if you want exposure to these three different types of customers.

reply