upvote
Without knowing any specifics of your uses, my usual starting point on that sort of design is that "messages AND data" is it's own special little way of ending up with a hard-to-debug-and-operate system. ;)

It's very hard to best-of-both worlds event-driven system + RDBMS-storage, it's very easy to end up with worst-of-both-worlds. Hello distributed transactions!

Again, you just should think about all the ways you want to use it and the maintenance/uptime requirements your users are going to have in advance.

reply
I think messages + database are extremely common in any sort of large application where you have data processing nodes. Postgresql actually has very good mechanisms to support message style communication, and as long as you design your message tables independently you shouldn't have horrid issues around locking and transactions. Message queues don't save you from thinking about that anyways, they just replace transactions with acknowledgements.

Trying to manage a highly available and durable rabbitmq or other message system that can also be recovered from backup to an offsite mirror infrastructure in the worst case is actually incredibly difficult. Usually these systems are designed with the assumption that you can just regenerate messages based on database state anyways in worst case scenarios.

In this use case your database already is highly available and can recover on an offsite backup if you have suitable wall shipping going on. So you've done all the hard work once, may as well reuse it unless you truly have some mind bogglingly large message throughput needs.

Finally, we had a need of a queue that was more than just first in first out. We wanted to fairly balance workloads across users and tenants. Whenever you have such a need postgresql lets you design this type of queue far easier than trying to do some elaborate multi-queue setup with a traditional queue.

reply
And now there’s pgmq
reply