At the time of Sonnet 4.6 it was probably the best choice. Haiku was pretty dumb back then. Luna didn't exist.
replySonnet 4.6 was released 6 months before Luna (and Luna's major price cut came a month or so after that IIRC). So it's likely they started with Sonnet long before Luna was an option. I assume they evaluated Sonnet 4.6 vs Haiku at that time and found Haiku insufficient.
replyAfter that, I'm guessing it was just fairly low priority to update a working system. Amazon's net income last quarter was >$20B. Saving millions is huge on the scale of one or two engineers spending a week tackling it, but it's too small for the organization to make it a big initiative. By the time engineers decided to update it, decision models like Jev were popular and are naturally a better fit for the task than an LLM.
I’m guessing accuracy. Smaller models are just THAT much more inaccurate to not make it viable.
replyI think for this purpose a small model will do reasonably well
replyWhat's weird is I didn't see mention of accuracy in the article.
replyNot really. Haiku is really not good at decision tasks (source: real experience).
replyRight. So we can either assume the engineers at Amazon are fucking clowns who don't do any engineering, or we can assume they tested and found that Sonnet was the cheapest model that met their reliability targets.
replyIf you think the former is a likely hypothesis, then why did you even click on TFA?
Anthropic was the only frontier lab on bedrock for a while. Hate to break it to you but there are in fact many scenarios with reasons and pressures for non engineering-driven decisions to dominate
replyLet's keep the discussion civil
replyThat's rather extreme? There is an possibility than Amazon's engineers are neither infallible nor idiots. Regardless of this specific situation highly talented engineers very often take decisions that lead to suboptimal outcomes on organizational level due to various reasons.
replyOne can think the circumstances a circus without thinking the participants all to be clowns.
replyAs for your question, seeing what others are doing helps one decide both what to do and what not to do.
Strands decider actually uses a small cheap/free Chinese model for its base architecture: Qwen
replyActually they started with Qwen 3.5, then switched to Gemma 4 since it was more accurate.
replyNot sure if it would even need to be a small Chinese model. Amazon would surely have the necessary hardware and infrastructure to run a bigger Chinese model with a low per-token cost too. They probably get their Sonnet at a discounted rate too since they do the inference "locally", but I imagine the savings for not having to pay Anthropic their share would still add up.
reply