upvote

points

by wongarsu7 hours ago |
comments
upvoteby tkz13127 hours ago|
next
[-]
Claude will write you something custom that fits your exact requirements with zero supply chain risk in an afternoon.
reply
upvoteby spike0217 hours ago|
parent|
next
[-]
how is there zero supply risk? claude will still package dependencies into a homemade solution at some level...
reply
upvoteby tkz13127 hours ago|
parent|
[-]
sure zero was a bit strong, but I personally feel pretty good about depending on only the go stdlib, compared to pulling in litellm.
reply
upvoteby Cu3PO426 hours ago|
parent|
prev|
[-]
Even if it plays out that way (which I doubt, we have a ton of requirements around spend tracking), the problem I'm seeing is long term maintenance. I don't want to fudge around with my proxy every time one of the model providers comes up with a new parameter that works slightly differently on Bedrock than it does on Azure.

That said, I am looking into alternatives. Two times in one year is really a terrible look.

reply
upvoteby CamperBob246 minutes ago|
parent|
[-]
"Claude, fudge around with my proxy."
reply
upvoteby dannyboland6 hours ago|
prev|
next
[-]
After the last LiteLLM compromise, I took my requirement (an OpenAI compatible endpoint routing across multiple providers based on latency) and built that as a small static Rust binary - https://github.com/sturnus-dev/sturnus

Now that AI makes it cheaper to build such tools in-house, the case for adopting a maximalist framework and trusting that entire supply chain is a lot weaker.

reply
upvoteby HumanOstrich3 hours ago|
parent|
[-]
> and built that as a small static Rust binary

I don't think memory safety would have helped here.

> the case for adopting a maximalist framework and trusting that entire supply chain is a lot weaker

You mean the Rust ecosystem?

reply
upvoteby hoppp3 hours ago|
prev|
next
[-]
I just installed it on my server in docker, then the UI was unusable and I deleted it and vibe coded a minimal alternative in golang. It works better.
reply
upvoteby connorbrinton6 hours ago|
prev|
next
[-]
Pydantic AI has a great set of abstractions over LLM providers. The documentation tends to focus on its agent loop more, but I find the LLM abstractions quite nice even on their own
reply
upvoteby TZubiri6 hours ago|
prev|
[-]
As with many dependencies, I wholeheartedly question the requirement and relentlessly cut them out without replacement.

Taking the definition from another user on this comment-chain:

"an OpenAI compatible endpoint routing across multiple providers based on latency"

If the endpoint is truly OpenAI compatible, you can just switch out the vendor by switching out the HTTP address, we are talking potentially about a single line change.

- LLM_HOST = "api.openai.com" + LLM_HOST = "api.competitor.com"

It sounds silly, but remember that software is a winner-takes all game, 99% of devs will lose, so if you are seeing 99% of devs do the same thing, it isn't surprising that it's a grossly wrong choice. It takes skill and guts to be in the 1%, reject the mainstream trends, and just write code without grasping at external imports to do even the most basic things.

I'd like to take this opportunity to sell out, I'm open for contracts if you are looking for a remote 1% dev. Early stage startups only, email in userpage.

reply