I have encountered this too. I am building a coding harness for www.propelcode.app and it was working really well until the claude code leak and then all of the sudden it seems almost intentionally stupid or outright manipulative in guiding me down wrong paths. At this point I am using other models for anything related to the tool use design and implementation and bought three mac studios with 512gb ram to run large open source models.
replyThis experience has made me feel like we have to create a community that moves AI from the mainframe era to the PC era quickly, or we will end up serfs.
I had Claude walk me through getting local LLM models running on my Mac a month or two ago and so far as I can tell it was intentionally helpful. I even stated the reason was to have an uncensored model for myself and it had no objection. Long story short LM Studio running a Heretic Gemma 4 is doing just fine on my system now.
replyI run a few local models for different things. I find Gemma 4 great for writing but qwen better for coding.
replyI tried the same prompt on gemma4 and qwen 3.5 and Gemma consistently failed to call the multi line edit tool.
I've had the same bad luck with tool-calling on Gemma4. Looking around the web, we are not alone. For other tasks, it's seemingly quite quick and decent.
replyBut it gets stuck in tool call loops, it seems like.
Oh to be clear I don't think Gemma 4 is suitable for real work. It runs at 10 tps and is somewhere between 4o and o1 in quality according to my subjective judgement. But Claude was happy to correctly tell me how to get it running and how to solve the pitfalls I encountered in that process.
reply