upvote

points

by kilobaud3 hours ago |
comments
upvoteby faxmeyourcode2 hours ago|
[-]
I aggree - I'm not understanding the value of the project either if you look at the example here https://github.com/microsoft/pg_durable/blob/main/examples/i...

It's an interesting technical achievement I guess, but it's very bizarre to try and read this

    SELECT df.start(
        @> (
            ($$SELECT ... FROM demo.invoices WHERE status = 'pending'$$ |=> 'inv')
            ~> df.if_rows('inv',
                $$UPDATE ... SET status = 'processing'$$
                ~> (df.http(...) |=> 'resp')
                ~> df.if($$SELECT $r.ok$$,
                    -- classify, branch, wait for signal ...
                ),
                df.sleep(5)
            )
        ),
        'invoice-approval-pipeline'
    );
reply
upvoteby rswail2 hours ago|
parent|
next
[-]
Without reading any of the doco, it appears to be a job definition called invoice-approval-pipeline that runs every 5 seconds.

The steps are:

1. Get all the pending invoices

2. Set their state to "processing"

3. Call out to an external service/process to do the actual processing, wait for a response.

4. If the response is OK, do something

5. Wait 5 seconds and then start again.

Not sure I love the syntax and the way SQL is embedded between the $$

But it is in the database, can be updated and modified in the same way as all the other stored procedures/functions, allows job control, I assume other control structures for parallel steps etc.

Gonna go read the doco now.

reply
upvoteby franckpachot2 hours ago|
parent|
[-]
"dollar quoting" is the PostgreSQL way to quote strings with quotes, avoiding double quoting or escape characters. I like to use the tagged version of it, like $sql$ SELECT ... $sql$ to describe what is inside.
reply
upvoteby gdecandia2 hours ago|
parent|
prev|
[-]
Contributor here - at Microsoft we've built AI workflows on pg_durable and seen it substantially reduce code and increase reliability. Agree that the DSL ergonomics can be improved. Our pipelines use a higher level language and therefore simplified, but pg_durable is meant to solve a wider array of problems. We're happy to take suggestions for improvements.
reply
upvoteby faxmeyourcode2 hours ago|
parent|
next
[-]
Somebody else in the thread brought up the benefit of snapshotting a database at a point in time stores not only the state of execution but also the code, etc. That is a unique benefit I'd be interested in exploring over storing your orchestration outside of the database.

Not trying to dismiss the project - it looks like a lot of hard work has gone in and somebody has a use for it. I just come from an airflow style external orchestrator frame of mind that manages durability state in postgres but keeps the control flow out. Sorry if I came off as a bit snarky

reply
upvoteby Onavo1 hours ago|
parent|
prev|
[-]
Do you plan to open source the high level wrapper too?
reply