> because of the overhead of parallelizing, despite them running at 100%

Duckdb might not be the reason, the real reason might be AWS.

It's been two years since I worked on this, but AWS Nitro might be triggering a high CPU Steal % when all the cores are busy at the same time.

You can do taskset -c 1-191 <server-process> and get a tiny bit of breathing room there.

I almost always do this above 32 cores, since this lets me always have a nice responsive ssh connection when the rest of the CPUs are churning high.

The more cores you have, the less of a problem sparing 1 core becomes.

This is also a problem in classical HPC, there aren't very many algorithms that can scale linearly with cores. That's partially why so many HPC algorithms tend to be built on matrix multiplication and Monte Carlo, because those do tend to scale near arbitrarily.