There are issues with the chat template right now[0], so tool calling does not work reliably[1].
replyEvery time people try to rush to judge open models on launch day... it never goes well. There are ~always bugs on launch day.
was just merged
replyIt was just an example of a bug, not that it was the only bug. I’ve personally reported at least one other for Gemma 4 on llama.cpp already.
replyIn a few days, I imagine that Gemma 4 support should be in better shape.
Qwen 3.5 27B is dense, so (I think) should be compared to Gemma 4 31B.
replyOr Gemma-4 26B(-A4B) should be compared to Qwen 3.5 35B(-A3B)
Exactly, compare MoE with MoE and dense with dense otherwise it's apples and oranges.
replyThe models are not technically comparable: the Qwen is dense, the Gemma is MoE. The ~33B models are the other way around!
reply