upvote
If you start here, with the "Postgres will take you wherever you need to go" meme, without thinking extremely deeply about your schema and how you expect to evolve it in the future, you can easily paint yourself into a very difficult and expensive corner.

It's easy to use Postgres poorly in ways that result in painful centralized bottlenecks.

(Obviously this is largely true for anything, but I think that in 2026, where there's also a lot of more-specialized/less-fleible but much-easier-to-scale well-supported mature alternatives, you should be VERY wary of making everything have a single central SPOF. What are your users going to expect in terms of maintenance windows, etc.)

I'd be cautious with articles that say things like "All cloud providers allow you to run (and scale!) PostgreSQL by clicking a single button." with no mention of how long that will take and what options should be set to make it faster, or the costs of those things.

reply
A counter anecdata. We transitioned from a postgres job queue to Rabbit. We had never ending problems after that, many of them were misunderstandings, some where just wrong-fit. We migrated because we had some time on our hands and thought we would alleviate some high pressure jobs. Not only did it not solve the problem, but having written all the code that decides when to pull the next message and what to do with it, and how to dead-letter it - just worked great for us on Postgres. It was so easy to understand and doing things like reprocessing just using a standard postgres DB interface was much easier.

Ultimate the entire processing got removed from our team and no longer needs to do these deployments (acquisition transitions)...

reply
I tried very hard to use postgres as a queue, it was robust but slow once I started to push from more than a few processes/servers. Moving to zeromq initially and sqs after solved all my perf issues, and was still solid.
reply
I think that’s the way to go. Start with Postgres and only if there are problems, then think about something more specialized. Same for microservices. Start simple and introduce a service when really needed.

I hate it when people already start out with 10 or more different systems/services for a few messages per second.

reply
I think that using the right tool for the job is important and saves a lot of time in the long run. There are expensive headaches that we have to resolve.
reply
it's not the way to go if you hit the limits very quickly and have to waste immense time migrating.
reply
Honestly Rabbit sucks more than it shines

Also it is very "unconventional". Everything has to be done in its weird and quirky way

reply
Having done that, e.g. used rabbitmq plus postgres, honestly I wish I had just used postgresql for both messages and data. It would have been easier to manage by an order of magnitude, especially at scale and needing to satisfy enterprise requirements. Also the flexibility of postgres would have solved problems that we ran into because of limitations of rabbitmq.
reply
Without knowing any specifics of your uses, my usual starting point on that sort of design is that "messages AND data" is it's own special little way of ending up with a hard-to-debug-and-operate system. ;)

It's very hard to best-of-both worlds event-driven system + RDBMS-storage, it's very easy to end up with worst-of-both-worlds. Hello distributed transactions!

Again, you just should think about all the ways you want to use it and the maintenance/uptime requirements your users are going to have in advance.

reply
I think messages + database are extremely common in any sort of large application where you have data processing nodes. Postgresql actually has very good mechanisms to support message style communication, and as long as you design your message tables independently you shouldn't have horrid issues around locking and transactions. Message queues don't save you from thinking about that anyways, they just replace transactions with acknowledgements.

Trying to manage a highly available and durable rabbitmq or other message system that can also be recovered from backup to an offsite mirror infrastructure in the worst case is actually incredibly difficult. Usually these systems are designed with the assumption that you can just regenerate messages based on database state anyways in worst case scenarios.

In this use case your database already is highly available and can recover on an offsite backup if you have suitable wall shipping going on. So you've done all the hard work once, may as well reuse it unless you truly have some mind bogglingly large message throughput needs.

Finally, we had a need of a queue that was more than just first in first out. We wanted to fairly balance workloads across users and tenants. Whenever you have such a need postgresql lets you design this type of queue far easier than trying to do some elaborate multi-queue setup with a traditional queue.

reply
And now there’s pgmq
reply
I once needed to maintain an application written in everything Oracle. If I ever encounter the original author of that product: I have things to say to him.

We quickly replaced part by part by easier, less costly parts.

Software development is not just writing code; I think all HN users know that.

reply
I dunno. I think the main takeaway here is that you can do 80-95% of your stuff in Postgres and eschew all the unnecessary, unproven stores.
reply
It’s also entirely possible that nothing you do in the eventual history of your company hits a scale where this matters.
reply
Sometimes to scale is "continue to satisfy SLAs as service usage increases" and other times to scale is "successfully evolve functional capabilities over time."

While the GP may have been referencing the former, embracing "PostgreSQL for Everything" often prohibits the latter.

reply
> in painful centralized bottlenecks.

I find the opposite to be true. I cut out the decentralization and get it all one one machine, and the bugs go away and the perf improves.

reply
It doesn't take very long (because compute and storage are separate in most of them) but good lord does it get expensive. Every time you click that upgrade button you are doubling your cost. It's really painful when you have a spiky workload that is performing fine like 95% of the time but you are watching the p99 and need to double the cost of a very expensive infra component, only to improve the experience of the heaviest 4% of your workload. This is to say nothing of the gambit you then have to play with reservations/prepays.
reply
I haven't seen a way to get guarantees of upscaling operations under like 30 seconds (with Multi-AZ RDS) with well-supported RDS stuff (leaving out active-active setups with logical replication because that's a whole other can of worms).

If you know you're gonna be ok with that for a long time, go nuts. I'm just saying: think about it in advance!

The cost pain for spikes is also a thing - some of Aurora's billing models look potentially promising but I haven't used them in practice - though it's also somethings that's harder to avoid with alternatives. Distributed DBs aren't generally super friendly to dynamic scaling IME.

reply
> If you start here, with the "Postgres will take you wherever you need to go" meme, without thinking extremely deeply about your schema and how you expect to evolve it in the future, you can easily paint yourself into a very difficult and expensive corner.

Yeah, backwards compatibility is not a thing for Java, Rust, C++, etc. :eye-roll:

Meanwhile in SQL if you need to make a backwards-incompatible change to your schema you can always use VIEWs and INSTEAD OF triggers to maintain backwards compatibility for code you've not fixed yet.

reply
As SRE dealing with this at current company, a benefit of using well known software like Kafka is a lot of problems you will run into have solutions/guidance already available vs you having to explore solutions which a lot of time end with “Kafka could easily do this. “
reply
100% except when Kafka goes wrong, who maintains it?
reply
There are two sizes of companies: those that can afford '1+ dedicated ____-person' and those that can't. Which should filter through to technology choices more than it does.
reply
Often you start as the latter and grow toward the former.

That transition can be super super painful as you don't quite have enough work for the dedicated person.

reply
I love the naming Kafka. Either they knew what it stands for or they didn't. And the latter is the worse option.
reply
This problem doesn't go away with postgres. It's totally anecdotal but this is one thing that I've noticed different in mysql shops and postgres shops - with mysql there is usually at least one person on staff who knows MySQL DBA and scaling pretty well, with postgres it's rarely the case to have someone who knows the internals well - like you said, the person capable of maintaining it when it goes wrong.

You could argue it's because postgres requires less poking though I would say you don't need the DBA for when things go right.

Of course most people are just handing the management off to the cloud and that's potentially why, but it doesn't cover everything

reply
MySQL will generally run fairly well with default tuning, assuming you've sized the buffer pool well relative to the amount of RAM you have (cloud providers do this automatically, but it's also not that hard to calculate). There are some knobs you can turn to eke out more performance in certain situations, and there are some defaults that are truly terrible (lock_wait_timeout is set to 1 year...), but all in all, it doesn't take a lot of care and feeding to run reasonably well.

Postgres, on the other hand, has a million knobs, many of them interact, you'll find conflicting advice for some of them, and it can rapidly fall over if you aren't keeping a close eye on long-running transactions. It's also more performant than MySQL in _most_ situations (hello, clustered index), if you've tuned it correctly. It also of course has far more extensibility out of the box, with tons of index types that are extremely helpful, if you know how and when to use them.

This difference is why I'm always frustrated when people parrot "just use Postgres" as though that solves all problems. It's an extremely powerful tool that can replace most of your stack, yes, but it also would really, really like you to RTFM. Not random Medium blog posts, the canonical documentation.

reply
> You could argue it's because postgres requires less poking though

This is the myth people who parrot "just use Postgres" believe. It is false, obviously

reply
I mean, with this custom thing, you have that question as well with downside is you cannot pick up the knowledge from off the street.

I became the Kafka guy at my current company, it took me about a week of reading and every time I had further question, I didn't have to bother anyone, I could Google and get data I needed.

When it's some NIH thing, you have to bother coworkers and knowledge is whatever is in YOUR company knowledge base with no ability to get knowledge from outside the company.

EDIT: You could also leverage contractors or outside support if not homegrown software.

reply
Yup, and every time it breaks, you've got to pester someone whose job probably isn't maintaining that thing actively.

I worked at a startup with massive NIH syndrome, once. We even used our own in-house programming language, because it was "better than anything else out there on the market." It did have a lot of nifty features that others don't have: a pretty novel type system, programmatic macros, a built-in build system and other fun bells and whistles -- but also not-so-fun ones like having no syntax highlighter, LSP, or debugger, and having to constantly shuffle around your code to avoid ICEs in the compiler.

The compiler wasn't the product, but we found ourselves fighting that thing more actively than any of the real problems our custom programming language was supposed to solve. The CTO found himself spending all his nights and weekends mostly trying to get the compiler to not explode.

A few years later, after I had long left (for that reason, among many) I heard they switched to Python. Can't imagine how long it took them to get that all rewritten.

reply
> e even used our own in-house programming language, because it was "better than anything else out there on the market." It did have a lot of nifty features that others don't have: a pretty novel type system, programmatic macros, a built-in build system and other fun bells and whistles

A DSL can work, but not for the features you list. Those features you already get from existing languages anyway!

If you need general programming language features like excellent type system, programmatic macros, a build system (doesn't need to be built into the language), etc... then use a general purpose programming language.

I have a DSL for backend/endpoints, and exactly none of those are in my feature list. What it has are things like easy way to specify access-control directives[1], the SQL query to execute, mapping request variables to SQL parameters, mapping SQL results-sets to response fields, etc.

I have another DSL for a test program. Both of those DSLs have specs that's literally 2x screens of bullet points and examples. LLMs can output those DSL programs because the spec for the DSL is so small.

For general purpose programming stuff (while loops, conditionals, etc) my DSLs break out to Python.

A good indicator that you shouldn't be creating a new language for production is when you find yourself implementing conditionals, loops, etc.

===========================

[1] Limit endpoint to specific roles, or members of the same team, or both, or even to the user itself - someone calling `/user/profile/update` should only be allowed if the profile they are updating is theirs, for example.

reply
Ah, but you see, it was a general purpose programming language. A general purpose, functional, optionally-typed programming language, with its own optimizing compiler and build system. In fairness, it wasn't originally developed to be our in-house language, but it was the creation of the CTO.

It was fun while it lasted and I had a lot of fun working on it. But it was really not a good business fit. The programmatic macro system was supposed to allow us to build customer-facing DSLs on top of it, but everybody just wanted Python anyways.

reply
> A general purpose, functional, optionally-typed programming language, with its own optimizing compiler and build system.

> The programmatic macro system was supposed to allow us to build customer-facing DSLs on top of it,

Honestly, it sounds a lot like Lisp.

As a former Lisper, I don't doubt that it was a bundle of fun :-)

reply
Nose goes
reply
During pgConf.eu in 2016(-ish, could have been one or two years later; I don't remember too well), a representative of payment processor Adyen told the audience that they were, essentially, one big postgres cluster in their backend, too ("cluster" used as per the postgres-native meaning of the term, as in, an installation on a single host with a data directory containing any number of databases).
reply
they likely have something on top of PG to distribute data across shards, which is still untrivial task I think and require ops overhead.
reply
starling bank uk uses a similar kind of stack. both java based as revolut.
reply