You can't trust those results no matter what
The pages that they pull in to source that data all contain affiliate links and companies contact websites to get their tools to the tops of those lists by paying money often monthly. I know this because I do this...
It's basically standard SEO but it also manipulates AI like ChatGPT very very easily
There are key differences.
1) Google doesn't get paid for the SEO, so even is crime is involved, Google isn't directly responsible.
2) AI ads are unmarked, which is illegal pretty much everywhere. And because of the way LLMs work, it is impossible to tell where a given output came from, neither which part of the prompt/context nor whether it's from the prompt or training.
Google doesn't get paid directly for the SEO but they definitely benefit monetarily. Do a recipe search and ask yourself if these are the results the user would like to see. Google benefits by not penalizing sites which litter themselves with ads. It's not that indirect.
They won't get you on any worthwhile list unless it's their own because it's too risky for them and any site they would publish it on would want to use their own affiliate link. Unless of course we are talking about something like Medium or YouTube which does work
And then of course there's the fraudsters who will bid on branded keywords we have banned dozens of people for that
But yes actually I was doing this about 15 years ago in the men's fashion subreddit for one of my companies lol
I don't think you can fine-tune your way out of it.
To filter bullshit it would first have to understand bullshit, and it doesn't. That's why an LLM will tell you the solution to a problem that doesn't work, and argue with you when you correct it.
For me, it's a resource wasting text generator. I'll not lie, I don't use OpenAI, Mistral or Anthropic's models, even for coding. I prefer to read my API docs and cry once.
I used Gemini, five or six times in total. Twice I asked a couple of very specific things, and it unearthed them. Since they were not products, but information, that was helpful. Twice, it has given wrong information. When I "told" it, there was another way, it said "of course there are two ways", etc. Tasteless and time wasting.
I don't like using an LLM all day long, or offload my thinking to them. It's the ultimate self-poisoning incident.
And as you say, these algorithms can't know right/wrong/logical/bullshit, etc. They just spew out text.
With LLMs, everything is given the same importance so you have no idea if the data came from a reputable source or an obvious SEO junk website.
Companies then get to bid for a preference “place”. This is more like Google paying to be the search engine default in Firefox.
And they are trained on web data just like any other model...
I'd be more worried about AI convincing you that you need a product or expensive solution when you actually don't.