upvote
> Will call Claude APIs when I get really stuck, but I should be able to handle 80% of my needs with a dumber local model.

I experiment with all of the local models I can fit into 32GB of VRAM and I have subscriptions to multiple SOTA providers.

The difference between them is very large, unfortunately. The local models can handle small tasks and refactoring mostly okay, but doing anything challenging with them becomes a waste of time. Unfortunately the waste isn’t immediately obvious because they will come back with something that looks like it works, but then on closer examination I need to throw it out and reset them in a usable direction.

reply
deleted
reply