upvote

points

by ivraatiems10 hours ago |
comments
upvoteby data-ottawa8 hours ago|
next
[-]
reCaptcha is a pretty strong wall to allow only Google to index websites, especially now that you need device verification. Throw in Cloudflare too.

There’s not much room to squeeze in when your competitors hold the keys to 15 million top websites.

reply
upvoteby xmcp1235 hours ago|
parent|
[-]
I write a lot of scrapers. Both of those are pretty trivial to bypass at scale.
reply
upvoteby HDBaseT2 hours ago|
parent|
[-]
What about not at scale?

I find it wild that "at scale" we can bypass anti-bot measures, but just "normal" internet use (i.e Non-Google Browser or VPN) will throw a million captchas at you.

cgnat is pretty bad too.

reply
upvoteby nostrademons9 hours ago|
prev|
next
[-]
The problem is that the web as we know it (useful, human-curated information that's put out there to help people) is also over. It's been totally overrun with AI slop. Even before AI could be used to create propaganda on a scale that we could only dream about 5 years ago, it's been declining under the weight of SEO sweatshops for a good 10 years. Meanwhile the actually decent content, the individual hobbyists who are just sharing their knowledge, have largely left under the weight of comment spam and DDoS attacks and doxxing.

So if another search engine does arise, it won't find anything useful, because the useful content on the web has been buried under slop, and largely removed. Your best bet today is a curated directory, sorta like the original Yahoo, where you allowlist the web to only real sites, download them, and make them searchable. I think this is actually Kagi's approach. But the open web as we knew and loved it is dead.

reply
upvoteby ares6236 hours ago|
parent|
[-]
Curators will become desirable again. The Devil Wears Prada 3.
reply
upvoteby torben-friis10 hours ago|
prev|
next
[-]
My literal first thought was "do I seriously need to use bing now?".
reply
upvoteby Supermancho9 hours ago|
parent|
next
[-]
Bing has been better than Google for some time. Again, it's embarrassing for them to sacrifice marketshare for paid results and an intermediate-form AI fad that will turn into the same paid result funnel.
reply
upvoteby mrweasel9 hours ago|
parent|
next
[-]
Bing is surprisingly not to bad. I don't use it anymore, but it's been providing better results than Google for sometime.
reply
upvoteby RyanOD9 hours ago|
parent|
prev|
[-]
I hear people cite other search engines as "better" all the time. Better how?
reply
upvoteby edelbitter8 hours ago|
parent|
[-]
e.g. for a two keyword search, Google & DDG return results containing a similar (but more at the moment, more popular, so I understand why they do this) keyword as the first one, and no relation whatsoever with the second. Any search that manages to actually show results related to both of my input terms get the "better" award from me.
reply
upvoteby sphars9 hours ago|
parent|
prev|
next
[-]
There is a 99% chance (IMO) that Microsoft is going to go the same route as Google here
reply
upvoteby endofreach9 minutes ago|
parent|
next
[-]
Also, usually, as soon as they realize they have a not-total-shit product, they immediately start to screw it up completely. So if bing ends up being better actually, it won't be long until they replace every good part of it with something ridiculous. I don't know how microsoft does it, but they are so incredibly good at that.
reply
upvoteby vitorsr6 hours ago|
parent|
prev|
[-]
Microsoft has already gone down this road some three years ago...

https://blogs.microsoft.com/blog/2023/02/07/reinventing-sear...

reply
upvoteby tedd4u9 hours ago|
parent|
prev|
next
[-]
DuckDuckGo uses the bing index/backend. I’ve had it as default for 5-8 years. Probably once a day I’ll add the !g to pop it over to Google. Works great. I search a lot, many different types of queries. When I pop over to Google it’s usually a Boolean query looking for a needle in a haystack (that one comment somewhere where someone is using the same combination of two or three rare items together).
reply
upvoteby BrunoBernardino9 hours ago|
parent|
prev|
[-]
While there are good options like DuckDuckGo, Mojeek, Ecosia, there are plenty of (better) alternatives, where you're not the product [1], I'd recommend looking into!

[1]: https://alternativeto.net/software/google-search/?license=co...

reply
upvoteby Zigurd9 hours ago|
prev|
next
[-]
I'm sure there's a niche for a product for search nerds. Something that leans into inverted indexes like the classic Lexis/Nexis search. But it's got to have Google-like coverage.
reply
upvoteby cortesoft9 hours ago|
parent|
[-]
Niche + Google-like coverage is not very economically viable. To store and update a search index of that size requires a lot of resources, and being niche means you don’t have a lot of resources.

Very few of the smaller search engines actually do their own indexing for exactly this reason.

reply
upvoteby edelbitter8 hours ago|
parent|
[-]
I wonder if the same coverage as before is now more economically feasible. The internet has gotten .. smaller, lately.
reply
upvoteby raincole9 hours ago|
prev|
next
[-]
Kagi relies on Google search.
reply
upvoteby baggachipz8 hours ago|
parent|
next
[-]
True in large part, but they've been diversifying their providers in the expectation that Google shut everybody out.
reply
upvoteby hootz8 hours ago|
parent|
prev|
next
[-]
But the results are still 1000x better than Google's. Something is being done there.
reply
upvoteby dgellow8 hours ago|
parent|
prev|
next
[-]
Sure but we are talking about the UI here, not the index being used
reply
upvoteby raincole7 hours ago|
parent|
[-]
But if Kagi manages to become a serious competitor in the search engine space, Google will cut them off from their index. Why will not they?
reply
upvoteby dgellow6 hours ago|
parent|
[-]
It’s possible but they would need to be so massive to even just start making a dent in google market share. And Google hasn’t blocked larger search engines from using their index
reply
upvoteby AndroTux8 hours ago|
parent|
prev|
[-]
They mostly use Bing, at least from my testing.
reply
upvoteby xerox13ster9 hours ago|
prev|
next
[-]
I've been using Startpage as my default search engine for a while now for any search where I actually need information and not sales or marketing bullshit.

When I use google, usually from my phone, I am reminded of why I don't use google on desktop.

With the announcement of this move by them, I just manually removed google as an address bar search engine option in all my browsers on desktop and mobile.

reply
upvoteby kylehotchkiss7 hours ago|
prev|
[-]
Cloudflare seems like they have the capability to take this on.

Human produced content should be separated from sites primarily hosting slop. That seems solvable?

reply