The intro to that post is so bad: "PostHog's original data warehouse had a single goal: delay the moment a company needs to hire their first data engineer."
replyIt is not to serve high quality and timely data to make good decisions, is to avoid hiring a specific role, lol. So bad.
Ironically a good data engineer should probably let them avoid needing 3 different data warehouse systems within a year.
replyAuthor here. I admit I kinda left out the context which is important here. We did not replace Clickhouse. Instead we want to move our warehouse product from Clickhouse to another stack which suits the long modeling jobs and multi-tenancy better. We learned through all our early design partners running huge workloads what we need to change.
replyAs much as I'd love to claim victory, we are not there yet :)
Engineers like to experiment and we get the benefit of learning from their experiences. Not sure why you’re upset.
replyno ones upset, I think given the quality of technical write ups we see on HN with very detailed analysis that this article felt like a faux tech article to promote a product? But it kind of is a self own as it just looks like they didn't do due diligence with their tech choices.
replyMaybe it's the "me first" mindset you're bringing to engineering?
replyThis sounds more like a 5 month POC that the engineer should have been able to prove sooner that DucksDB didn’t fit the use case.
replyAuthor here. We went past the POC stage to actually make real customer workloads work, which we did. Probably yes, we spent too much time not rethinking our approach but sunken cost fallacy so that's why.
replyI mean, mostly it just comes off as egregious self-indulgence to try one thing and then jump to "we built the solution ourselves!" when the problem is not novel at all. These things are just tiresome.
reply> Engineers like to experiment
replyThe job of management is to help manage that. Typically, the desire to experiment is constrained to lab environments and so on. Experimenting on your actual users carries a lot of risks - to product stability, customer relations, market perception, etc.
This is the same company with intentionally annoying full-wrap bus ads on Muni. They are trying to capture mindshare.
reply