upvote
Pretty sure your phone assistant isn't using their best model. It's probably using one of the old flash models no?
reply
The model used is irrelevant when there's the critical issue where their "AI" assistant can't understand the word "nearest". Literally worse than a second-grade understanding of English.

I also have had shockingly bad results with gemini on android. Ended up giving up on it after a few months. Silly silly google with their trademark UX polish and attention to usability.

reply
When you ask stuff in your car on android auto it uses 3.5 flash-lite. The response details would include it in the logs.

Flash lite is very noticeably more idiotic than 3.8 Flash. It's why the quick AI overview is often much worse than the AI mode one.

reply
As an android user, this worked perfectly like 10 years ago.
reply
Yes, there's been a huge degradation in the reliability and usefulness of Google's digital assistant following its incorporation of "AI".
reply
It's really puzzling. I was really happy with the original assistant, now it feels slower at literally everything.
reply
This reflects pretty much every result of mine from Google these days. Not just navigation, but search too.

The only thing I can figure is that Google's consumer-facing free-to-use models are set to "reason as fast as possible with only the most basic plausibility checks." I have to correct them like this every single time.

reply
Genuinely curious: why do you use them then? It's not like there is no competition in this area.
reply
On Android, the public APIs available for digital assistants are more limited than the ones that Google gets to use for their own products. So if you want a digital assistant and you'd like to be able to hold a button on your phone and have it open a thread with an LLM (even if your screen is off), your digital assistant has to be Gemini. You can configure ChatGPT or Claude in its place, but then they can't manipulate or interact with apps in your phone very much. Or you can use an app like that for all of your LLM use but you can't conveniently summon it.

Similarly, you can run local LLMs on your Google Pixel smartphone, but if you want them to be hardware accelerated using the chip that's included for that very purpose on your phone... it has to be Google's models via Google's apps. So you can't say "alright, a fast, local model which sounds great, but I want one that is trained to always run a search before answering, in a harness with good search tools for answering the kinds of practical questions I often ask", you're actually not free to put that together in a practicable way.

So there is indeed some competition here. But it's not really fair competition.

reply
I skip past them to the actual search results which tend to be much better.
reply
Uh, yes, of course. search results are fast & cheap & free, why would they use the good models?
reply
The first law of robotics prevents it from facilitating regular ingestion of low-quality nutrients.
reply
Gemini Flash is very dumb and gimped. Recent experience:

1. I ask it to list out all my old Keep notes that are just throwaway task lists, reminders, etc. to propose deleting them.

2. It comes up with like 4 of them (I know for a fact there are dozens, if not hundreds). I ask it for an exhaustive list.

3. I see its thinking trace briefly show "Refusing to produce an exhaustive list..." and then it says "Yes, that is all of them."

Not sure whether it's lack of intelligence, bad tools, or both, but that was extremely frustrating to see.

reply
My Gemini app on Android does not even have an exact location permission you could give it, so I'm not sure how you think that would work?

If it had exact location access people would be crying that Gemini steals their location data.

reply
this is like that tiktok guy who asks the spoken ChatGPT model very simple questions, like 'count down from 60', or 'how many e's in seventeen' or whatever and the model shits the bed every time it's hilarious

I think these companies have good models but expose them poorly like others have mentioned

reply
Often it seems like the problem with the Gemini apps is the harness, not the model itself. For example if I upload an image PDF it recognizes text worse than if I use the API.
reply
Agreed! I feel that they're pouring all of this work into models and then squandering that work by integrating them into bad products.
reply
Is this from you car ? Gemini in car is probably less intelligent than a spider.

I tell it to "Play Let it go from frozen". It plays it on youtube music. 5 seconds later I tell it again 'Play let it go from frozen" it then just dies saying "I can't play videos in car".

reply
The auto version is extra horrible because what it reads to you is just a summary of what the model actually said. I don't ask it anything when driving because it's too frustrating.
reply
Yeah, whatever they have running on Google Maps is probably a pretty low tier model, but it is total trash. I’ve only used it twice though and that’s when it first came out. I don’t know if there have been any changes since then. The main problem is I felt like it was not keeping track of the context of the session.
reply
Gemini models are in fact very good. Google's harness(es) for Gemini are nightmares, terrible
reply
ngl gemini is helping to provide (real helpful, not the ones giving runarounds) customer services in one of my own products.

i benchmarked 20+ models in real worlds and gemini families are the best (highest customer satisfaction)

reply
It sounds like the tool call is probably not grounded with your precise position.
reply
Sounds like it might not have your precise location in context?
reply
which is more or less the crux of the problem android auto's gemini implementation. It sometimes does not even know that it is in a car. The ghome implementation is also like this.
reply
I hate to be that guy, but Tesla's Grok doesn't seem to have that problem.
reply