As someone who helps keep a site online with a lot of content, I have mixed feelings on Firecrawl.
replyOn one hand, their bots seem much more well behaved than others.
However, running a crawler fleet which is deceptive and evasive in its identification and don't honor REP is no way to build a business.
fuck firecrawl. they copied my idea by showing interest in my product and then copied it, used their YC money to give it all out for free. fuck nick in particular. I'm still salty over this
reply"they copied my idea by showing interest in my product and then copied it". What exactly is revolutionary about Firecrawl or your product? Scraping APIs have been around for over a decade.
replyI was the first to return markdown and use reader mode stuff to strip irrelevant stuff. Theres copying and there's talking to the founder sounding interested to have your team copy what I did in the background. One is fair game, the other is a dick head move.
replyNot sure about the first claim. But yes, talking to the founder, sharing details and having it stolen is not a good look. Sorry that happened to you.
replyTell more. Crawling is not a new idea. How did they abuse you?
reply