Was thinking today... not a debugger but even a SQL progess bar, so I know that my add column will take say 7 hours in advance.
replyThis is awesome! I’m work with a team of analysts and data engineers who own a pretty big snowflake data warehouse. We write a ton of dbt models and have a range of sql skill levels on the team. This would be the perfect way to allow more junior devs to build their skills quickly and support more complex models.
replyI would recommend you target data warehouses like snowflake and bigquery where the query complexity and thus value prop for a tool like this is potentially much higher.
Thank you, nice to get some idea validation from folks in the industry. For sure data warehouses are the top priority on my TODO list, I picked PG first because that's what I'm familiar with.
replyI can ping you via email when the debugger is ready, if you're interested. My email is in my profile
This would be incredible to understand why some queries execute slow; most of the time it's one of the steps in between that takes 99% of the execution time at our company. Do you record the time each step takes?
replyCan you not use EXPLAIN ANALYZE to identify steps that had the highest compute time? I think most databases have some form of this.
replyThis is a great command everyone should know. We once had a long running database query that was blocking a pipeline (code was written in a week and of course became integral to operations). Ran it, 15 minutes of thinking, added a new index on an now important column, and cut the run time down from almost 30 minutes to 5 seconds.
replyI've never heard of this, and I'm pretty sure my coworkers haven't either. Thanks for mentioning it!
replyhttps://chatgpt.com/share/68104c37-b578-8003-8c4e-b0a4688206...
You're onto the original idea I started out with! Unfortunately it's very difficult to correlate input SQL to an output query plan – but possible. It's definitely in future plans
replyMSSQL has the execution plan thing that will tell you which steps are involved and how long they take.
replyPossibly look at https://duckdb.org/community_extensions/extensions/parser_to...
replyEven if not for DuckDB, you can use this to validate/ parse queries possibly.
Thanks for the suggestion! I am using https://github.com/tobymao/sqlglot, which magically supports most SQL dialects. And yes, support for DuckDB is also in future plans
replyThis seems like it could be extremely useful.
replyThanks! Would you mind sharing what would be your use cases?
replyAt my job, all of our business logic (4 KLOC of network topology algorithms) is written in a niche query language, which we have been migrating to PostgreSQL. When an inconsistency/error is found, tracking it can take days, manually commenting out parts of query and looking at the results.
Am not the person you asked, but feel that it could have good value for education and learning as well, besides debugging.
replyFinish it, shut up, take my money! This looks really good - make a website just to make it possible to sign up for updates.
replyThanks for the motivation to finish this as soon as possible :) I'm working on a basic landing page with screenshots/videos and a "get notified" button right now – shoot me an (empty, if you want) email (in profile) and I'll ping you as soon as it's ready
replyhttps://dequery.io :) Added a little signup form (with possibility of providing additional feedback)
replyThat was quick.
I wanted to be quick too, but learned that I was too eager and missed that the are optional fields to pick from. I then re-submitted with the same email (will it go through or you have uniqueness validation?) providing optional stuff this time.
Maybe play with positioning of the optional fields so they would be more apparent before submitting email address.
replyThanks for the feedback! I updated the form
replyCool! We're dealing with many complex CTEs and costly queries. Would be useful to have those visualized one by one.
replyWhat database are you using? I'd be happy to hear about your usecases and hopefully help you, shoot me an email (in profile)
replythis is very cool! Where can I follow you to see updates?
replyIs this postgres only? What an interesting idea!
replyFor now, yes, but I'll start working on adding support for all other DBs (especially OLAP) as soon as possible. The geberal approach is the same, I just have to handle all the edge cases of the SQL dialects
replyhttps://dequery.io
replyThis comment is still (surprisingly!) gaining traction, so leaving this here
Nice one!
replyFollowing...
reply