For real judgements on database products, I'd look to https://jepsen.io/. That author tends to be refreshingly straightforward in a field where many vendors love to cheat by different names
You don't have to take my word for it. If you want you can inspect the binary yourself: https://pgrust.com/downloads/archive/2026-09-clickbench-snap...
You can run the binary with PGRUST_GRAVITY_REPORT=1 which logs the code path hit for every query. You'll see that all the code goes through the general query path. We did have hand-rolled code in the binary, but we used it for testing. The code is not used when running the benchmark. We also did not use a results cache. Warm runs still scan all the data. You can verify all of this
You are right, though, that it did support a narrow set of SQL. It supported scans, group bys, sorts, and filters. That's enough to run all the queries on ClickBench, but failed on functionality not tested by the benchmark like subqueries
You keep giving this same answer to no one's satisfaction. Saying go check the binary is worse than saying go audit 400k+ files. How do we know the Ai has not cheated in the code path logging? How do you know that it has not?
The more you make excuses, the more we get suspicious
(copied and pasted from another place you gave this unsatisfactory answer)
It instruments the files that were called out in the PR, then runs ClickBench. You can run it and see that the code in those files doesn't run.
If you can find anything suspicious in the binary, let me know, and I'm happy to dig in
this would be less dramaful if you had said something more like "claude cheated and we didn't notice, here's what we will be doing going forward..."
people empathize more with those who admit mistakes than those who try to argue around them
What happens in an environment of cheaters is that cheating becomes so commonplace, the competition becomes one that assumes cheats are enabled, and then everyone involved either denies cheating or is silent about it because IYKYK and if you DK, you don't need to K. Look at bodybuilding, which is full of men (and women!) absolutely juiced to the gills getting on Instagram and assuring everyone that their physique is totally natty, bro, just hard work and proper nutrition.
Programming is headed in that direction. LLM use, even to the point of "vibeslop", is the new default, and while LLMs do overindex on benchmarks, our benchmarks are going to end up overindexing on validating whatever public opinion thinks is "great" at the time, irrespective of whether it actually is, so "cheating" as we know it will go undetected, and certainly unpunished if it is detected. I don't know what the equivalent of falsely claiming you're natty is. Maybe saying "I wrote this" when you wrote it about as much as the Pope painted the Sistine Chapel fresco?