upvote
Thank you for escalating the arms race that's currently destroying the internet. I can't look at the Apple user forums, my health insurance company's portal, pay my county taxes, browse Digikey and Mouser, view Github or whatever forge, etc. etc. without having to jump through Cloudflare or Anubis garbage designed to block your abusive behavior.

I'm just overflowing with joy that you're comfortable wasting my time and money.

reply
I use the automated captcha solvers, stealth browsers, proxy IPs and so on. In regard to ethics, I think this is the right thing to do. The minimum I can do is just to show this is not a matter of excuses or mental gymnastics to trick myself into thinking it's fine.

Even if you said, a lot of the public commons is destroyed, I would NOT use bots and cause more destruction because 'everyone else does it' or 'it's a lost cause'. I say it is the right thing to do because the balance is such that using the internet is actively harmful to the user. That is, using the internet with bots is neutral to the user, and using it without bots, you are being actively harmed and exploited.

If you are using search, this is not a library. You are using your time to train targeted advertising that exploits you; you are selling your search history to the highest bidder. If you are using ChatGPT you are just working as a teacher for free.

On services, you have paid for the product, but the seller is scamming you by not giving you the product. The seller is using dark pattern UI to trick you.

If you are on social media, don't think of this as a public park. You are being treated like a lab animal, your behavior is being studied for the benefit of a corporation.

So the point about the bots is that I think people should be empowered to protect themselves from the internet, which is otherwise actively harmful to their wellbeing.

reply
Even if it's not the GP, if there's profit to be made, somebody will do it. I was reminded of the power of economic motives the other day, when it turned out someone bought an entire trustworthy reseller just so they could compromise the Ledger supply chain by including a tiny eSIM module that OCRed the screen and transmitted the seed.

Incentives are crazy powerful.

reply
> if there's profit to be made, somebody will do it

Not always! With sufficiently powerful barriers it can be blocked at least in part.

Shaming awful people is one of available tools.

Or at least not praising them for making small profit at huge costs to others.

reply
Can't argue with doing what we can.
reply
deleted
reply
"Usage records — which endpoint was called, when, the credit cost, the status, and the target URL."

I would use it, but then the privacy policy says that you would store my search history. I simply want to use the internet in peace without being constantly recorded.

reply
hey,

I didn't get your point, how would you do design our usage records then ?

reply
Ideally it would be private.

Store: Customer ID, billing details, services/credit used.

Try not to store: The user's search queries, the user's prompt/response content, the user's web history (URLs, etc).

To avoid selling the llm prompt content you would need your own inference or a contract.

Sadly most just hoard the data and some will offer a model where they charge for privacy. They are all quite scammy.

Firecrawl: logs. Parallel: logs. Bright data: logs. Kagi: logs for abuse. Bocha, querit etc: logs logs logs. Cloudflare: doesn't say what they do. Exa: charge for no retention. Linkup: no retention on request. Brave: no retention on request, but actually approved it (win for brave).

reply
Here’s what is says on their site: “Courtesy backoff Adaptive backoff keyed to target response codes. When a target returns 429 or 503, we back off immediately, so our fleet never applies denial-of-service pressure to a site.” By the time you get the error you have already crashed the site. Geez.
reply
Anti-fingerprinting is mostly there to be able to use the same browser across many requests, I assume?

I found that a slightly-modified firefox for more stealthy remote-control works fine and gets a pass on all automatic captchas I tested. but I don't do it at scale, I just wanted something to automate a particularly stupid website that I have to use.

Would love to see if my browser is detected -- any recommendations for tests to run?

reply
I have to agree with the (sarcastic) sibling comment. You’re essentially making everyone’s life worse in the name of personal profit.

Please stop. The last thing we need in the world is more greed and selfishness.

reply
Since when have moral appeals worked against the central feature of capitalism?

Only strong regulation can help.

reply
Shaming awful people is one of available tools.

Or at least not praising them for making small profit at huge costs to others.

Also, it is useful as coordination method, enabling passing laws outlawing some behaviors.

(and I am not able to legislate-via-http while posting such comment is easy)

reply
deleted
reply