Very cool, it’s insane to me how frustrating the S3 UI is. My current annoyance is bulk downloading is hard, and seeing things like “how large is this bucket or specific key prefix in bytes”
replySaw you using DuckDB at internal/handlers/api.go
Wondering if you’d considered letting queries operate at the bucket or key prefix level instead of requiring key to specific file? Or is that already supported?
I’ve liked being able to point DuckDB to a folder following Hive partition scheme, and then querying it just like Spark/Trino would over many files
Thanks! Its true, many things are frustrating, like you say - bulk downloading. Yes DuckDB is amazing, and I use it in more than one of my projects. It has a lot of use cases for the data lake.
replyI agree, it would be more beneficial to query on patterns, but the main use case I saw for this app was non-tech folks wanting to see data in s3. I assume they wouldn't do fancy stuff like querying multiple files. But maybe I should add that functionality. Thanks for the suggestion!