Kagi had a post discussing this which made the front page of HN about a month ago [1]:
> Google does not offer a public search API. The only available path is an ad-syndication bundle with no changes to result presentation - the model Startpage uses. Ad syndication is a non-starter for Kagi’s ad-free subscription model.
> Because direct licensing isn’t available to us on compatible terms, we - like many others - use third-party API providers for SERP-style results (SERP meaning search engine results page). These providers serve major enterprises (according to their websites) including Nvidia, Adobe, Samsung, Stanford, DeepMind, Uber, and the United Nations.
> This is not our preferred solution. We plan to exit it as soon as direct, contractual access becomes available. There is no legitimate, paid path to comprehensive Google or Bing results for a company like Kagi. Our position is clear: open the search index, make it available on FRAND terms, and enable rapid innovation in the marketplace.
See this previous discussion: https://news.ycombinator.com/item?id=46708678
[0] https://www.404media.co/email/4650b997-7cc3-4578-834c-7e663e...
I see the same phenomenon on other smaller forums, too, though. DuckDuckGo always feels like it has a smaller database than Google, which isn't really a surprise.
I much prefer to use scholar.google.com or npmjs.com for research. The URL is already in my history/bookmarks and the scoped query is more useful than the generic websearch.