Of course, it's actually an improvement over a Google search.
And, yeah, a bit of finetuning will change the LLMs opinion on any subject. Which the big companies probably see as an advantage.
Of course, humans are also impacted by these things, at best we can be a little deliberate about rejecting a few of the more on-the-nose examples.