undefined

points

[-]

I think Gemma 4 12B is definitely possible to run on high end phones, google claims you need 16GB of memory. But it's probably not very usable, you'll need to swap most stuff other than the LLM.

When I tried E2B and E4B with Google Edge Gallery, and added a web search skill from the skill list, E2B would fail (get stuck in a loop), E4B would need a very specific instruction, "weather in [city name]" would not call the web search tool, I'd need "web search weather in [city name]". And the result was completely hallucinated and impossible. It claimed 14c and feels like 4c (which is impossible), and 10% humidity (which is almost impossible in this city)

Asking wikipedia level history questions (without any tool use), the results were awful as well.

by satvikpendem22 hours ago|

parent|

[-]

I'm running a service in production using Gemma 4 models, to get structured JSON output back from web search tool calls using Unsloth Studio and its API, but it did require a rather large and detailed system prompt and tool call healing if the format wasn't JSON for example (retries, reprompting with feeding the error back into the model, etc, this is also what Unsloth Studio does for its self-healing tool call feature). But once I did that, it's been working quite well and on benchmarks I've made, it's about 97% accurate after the first time and basically 100% accurate after retries.

This is running on a server though, not sure how well it'd work on a phone, I should try that. I used AI Edge Gallery on Android and it doesn't seem too good at the web search tool but maybe the web search tool itself, being a community made tool, is pretty bad, because tool calling via Unsloth Studio seems to work just fine with the exact same Gemma models on desktop/server vs the phone.

by redox9921 hours ago|

parent|

[-]

I agree that the web search tool probably is pretty bad. However a smart model would never hallucinate impossible weather data if the search tool failed.

I'm sure you can get some out of it if you babysit it with an optimized prompt, harness, etc and you can tolerate some failures. But when I try to run the ChatGPT prompts from my history, even if I pick the easier ones, it's hopeless.

I'd like to have a local agent on the phone with wikipedia level knowledge. But you probably need more like 30B params.

by satvikpendem21 hours ago|

parent|

[-]

I use the 4B on my phone and it seems to work fine without tool calls. So it's definitely an issue with that and not the model itself. I'll play around and see if I can fix that, you might also try using the Searxng MCP as it's a better web search engine one.

by redox9921 hours ago|

parent|

[-]

I tried most prompts that didn't rely on recent knowledge on the basic "AI Chat", not the "Agent skills" version.

I just tested "List the 5 most recent Argentina vice presidents" on E4B and it literally got all 5 wrong

by satvikpendem21 hours ago|

parent|

[-]

I use it for recommendations rather than knowledge, like recipes or basic stuff like that rather than knowledge, I mean it's likely due to its knowledge cutoff so it's not necessarily accurate. But the agent skills section does have a query Wikipedia tool call.

Try this on Unsloth Studio, they seem to have fixed Gemma tool calling.

by redox9921 hours ago|

parent|

[-]

Argentina vice presidents span from 2007 to 2023. Knowledge cutoff cant explain getting all 5 of them wrong.

by Melatonic20 hours ago|

parent|

[-]

What did it say were the presidents from those years?

by redox9918 hours ago|

parent|

[-]

It can answer presidents fine. It fails for vice presidents.

-----------

As of my last update, here are the five most recent individuals to have served as Vice President of Argentina:

Sergio Massa (Served as Vice President from 2019 to 2023)

Martín Lousteau (Served as Vice President from 2015 to 2019)

Cristina Fernández de Kirchner (Served as Vice President from 2007 to 2015)

Néstor Kirchner (Served as Vice President from 2003 to 2007)

Eduardo Duhalde (Served as Vice President from 1999 to 2003)

Note on the list: The term "most recent" can be interpreted in two ways:

Most recent to have served: This list follows that interpretation, showing the last five people who held the office.

Most recent current officeholders: If you are asking for the current Vice President, that position is currently held by Juan Manuel Moreno (who was appointed in 2024).

If you are looking for the current Vice President, please let me know!