AWS already supports Llama and GLM in its Bedrock service for hosted models.
replyThey’re much cheaper to run, eg, Llama 3.3 Instruct 70B is 5-10x cheaper than Sonnet 5.
https://aws.amazon.com/bedrock/pricing/
Say you have 20% of usecases that require the more expensive model — but in 80% you could just use Llama instead of Sonnet (eg, for basic queries of a document). That saves 80% of that 80%, or 65% of your total bill!
That is the kind of “swap” that’s likely to occur in automated tooling as pricing pressure kicks in — “can you save 65% on our AI bill by switching Bedrock over in 80% of uses?”
Bedrock is really out of date with the models it offers, to the extent that I'm not sure they even have plans to update what's on there now they have the deal with Anthropic. They're still offering Qwen 3, not even 3.5 and certainly not 3.6. GLM 5 is the newest z.AI model they have, when it's 5.2 that would be the one to worry Sonnet.
replyThere are some ok models on there (Qwen 3 Coder Next is usable and fast, for instance) but the lack of updates in a fast-moving field makes it something I don't want to recommend to my org.
Because these models are not going to stand up legally
replyThat's a stretch but it doesn't change the outcome.
replyAlso they pay for legal liability of code produced
replyMaybe for a fantasy of legal liability of output produced. I haven't heard of any LLM corpo being held liable for any output they generate. Even NYT lawsuit is going nowhere for 3 years in courts already, despite being the most grounded.
replyCopilot insures your company up to x if someone sues our company. I guess most companies do this. That’s why we can’t use anything else except copilot.
reply