upvote
That is NOT my experience. I used to have a dump of screenshots where Google AI from search is absolutely wrong. In my experience, it's doesn't work and "quoting" things never told in the source (the exact contrary sometime), responding in hindi, responding in spanish, responding in mix of languages.

Most of the time it's quicker, sometimes it's just bad.

reply
The argument is not that Google's AI summaries are infallible.

The argument is that there's so much garbage on the open web that clicking through to the source has a lower signal-to-noise ratio than the AI summary.

This rings very true in my experience.

Google's AI summaries definitely make mistakes, but in most cases they're higher signal than the source websites.

reply
You're underestimating how obviously, laughably wrong the AI summary often is.

I really think Google is making an error using such a lightweight model for the page. That's one of my only firsthand exposures to genAI, and it's awful. Use a decent model, or if that's too expensive don't put up a summary at all.

reply
They used to prioritize accuracy first, then speed second.

Now it's just speed. This is a grave error.

reply
It hallucinates so frequently, that I've found it unusable. Just ask it to provide sources for all of its claims, then watch how many times it reverses its original answer. It's mind-boggling to me. I've repeatedly tried to give Google AI Search a chance, but after weeks of seeing this behavior I can't trust it anymore.

Imagine winning search by being the fastest and most accurate, then replacing it w/ constant hallucinated errors.

reply
I think they model the model they use for these, probably at more cost to them. They've been much better as of late, imo
reply
They what the model?
reply
The model they model to model this is modeled by a model they model.
reply
For me, I use it maybe ones or twice a month when my traditional search on DuckDuckGo fails. And, anecdotally, it is overwhelmingly wrong, and worse, confidently so.

Given my sample is biased as I am using it for an outlier of difficult web searches (including web searches localized to Iceland). But I would think that if your parent was correct, at least majority of my uses would be correct, but in my experience, Gemini fails most of the the time traditional web search fails. So to me at least, it is effectively useless.

For people who use it in place of normal web search, it is probably worse then useless, because of how confidently wrong it is.

reply
Why would a summary of unreliable blogspam results be more reliable than the blogspam results themselves?

I find it easier to recognize bullshit from its source vs an LLM's confident and well written regurgitation of it.

reply
I consistently get the impression that Gemini is better at sorting through the trash than whatever ranks SERPs these days is.

(Obviously far from perfect, often far from even good... just better.)

Which is not a good look for the Search teams/execs.

reply
Only if the answer is already on Reddit.
reply
The summary isn't limited to the top 10 search results as sources so it need not be only based on blogspam like the first page of search results is/was. Whether or not it's actually better or not is still its own debate beside that point though.
reply
Because the summarizer knows what blogspam is.
reply
Absolutely. Google has shown the courage to admit that their bread and butter business is changing permanently. they are adapting which the large cos rarely do.

My mother never used search. She found it too hard to manage. She uses Gemini all the time, it gives answers in her language and she does not need to type. She is incredibly happy.

My daughter talks to Gemini. The idea of going through links seems primitive to her.

The future of interface is chat/talking not going through clickbait.

I think schools should have some basic control on how kids can use AI but saying that they want to freeze the learning experience in 2020 is simply not right. It is like saying we wont allow whiteboards and markers.

reply
Google shouldn't be making that choice for everyone.
reply
Thankfully, no one is forced to use google.
reply
I haven't checked this month, but the few times I used Google Search this year I saw egregious errors in the AI summary ~50% of the time. Things like confidently citing a statistic whose number appeared in the top 10 results, but misrepresenting what it pertains to.
reply
From my experience it's worse than any of the normal Gemini models
reply
The AI summary is just summarizing the top results. It’s not Gemini.

So I can’t understand how one would trust the summary but not what is being summarized.

reply
Not my experience at all. I can usually trust chatgpt or Claude to give me accurate answers for queries but google summaries are frequently wrong
reply
A recent study from oumi.ai found that the summaries contradict the sources they cite 10% of the time.
reply
Anecdotally, the percentage is way higher than 10% for some kinds of searches. I don't always use google, but i've tried it dozens of times the past few weeks to test their AI Summary / AI Overview, and it has yet to get anything correct. I've had to use other search engines or AIs literally every time.

mistakes vary from "obviously wrong" to "looks vaguely correct, but the source states something different" to "looks plausible, but the cited page is completely irrelevant, the AI clearly made this up" Even for straight fact lookups, I've seen it cite pages and claim one number, and then i click through and the source matches the vague shape of the answer but has a different number.

Its just constant hallucinations. "thats a common problem, just change this <setting that does not exist> or set <option that does not exist>". Full gemini seems more capable, but whatever google is throwing at the top of their free searches is awful IME.

reply
Use Kagi, it mostly excludes spam.
reply
Google is trying to behaviorally condition children.

The relative effectiveness of the search is immaterial.

This is gross.

reply
All we’re doing is adding another layer between us and the information. Google flash is still pulling on the same poor information. But it’s not as good at discerning what is useful and what is not as we are (people going way too far out on a limb outside of their domain expertise not withstanding, which is another part of the problem and why this stuff can be dangerous)
reply
Can’t access the article, but would assume the schools are more concerned about students having Google AI doing their work for them, rather than concern about using the AI summary for search results?
reply
Yes, this is the concern. Thankfully Google has a 'helpful' AI chatbot / tutor product that students can use instead of search!
reply
at least before I was able a find a blog post or a discussion or fleck of documentation. Now I've pretty much stopped using Google because I've wasted so much time having been assured by Gemini that something is entirely possible when in fact its very much not. now I have to dig to find the canonical reference for a question, at least SEO people didn't spend a lot of energy trying to front rust crates.

This is all very tragic. we built a magic network where we could stash all the worlds information and access it from anywhere, except its been trashed by people with sleazy motivations trying to get between us and what we'd like to see.

reply