What a nice project. What inspired this initially?
replyFYI there's a broken link in your readme:
https://rumca-js.github.io/internet full internet searchthanks, I replaced it with a other link demo
replyCant you just request the ICANN’s zone files and have the canonical list of the day?
replyAny link list, or domain list is not worth much without any rating, or meta. I lead a hobby project, and I am not expert, so I provide ratings based on what kind of data pages provide (title, social, description), and my own manual voting system. It is not ideal, but it is something. Also I provide tags, so it is easily known what the domain provides, or domains can be filtered by tags.
replyI know that you cannot count and visit every domain, so the list will never be finished, but I am happy with the results.
Avoiding GIGO (Garbage In, Garbage Out).
replyThis is why we have computer-variants of Library Science and Archeology, Forensic Science and a bunch of other advanced knowledge (not AI, mind you).
I don't see how this applies as its aggregating a bunch of stuff from random crawlers - if you want to crawl a list of actual domains that's generally considered the list of things that could resolve, so seems like a good starting place.
replyThis is amazing. Thanks for sharing!
reply[dead]
reply