It's also interesting how Clickhouse / Starrocks can now also act as a query planner and executor on top of non-native formats (ex. Iceberg).
replyI assume the native formats will always be faster / more optimized but the need for Trino as a separate executor while running either of these databases seems to be close to gone.
The benefit of StarRocks/ClickHouse over Trino is that you get secondary indices, but that means you have to do the indexing somehow.
replyNative format is faster (especially for colocated joins), but it's way more expensive if you have to run a bunch of separate storage nodes vs just using S3, especially your query volume isn't that high.
I liken it to the BigQuery cost model, where storage is effectively free.
I’ve historically read this as ‘open format compatible’ but ‘native preferred’ - where this opens up market space and dev velocity - but it’ll be interesting to see if native storage differentiation gets dumped entirely. It just seems like ‘fork and optimize for our engine’ would always be tempting enough that you’d want a native play for when you don’t need the decoupling.
replyClickhouse joins have been improving almost every month for the last couple of years. Maybe they still suck but a lot less
reply