Could be the bundler re-resolving the whole dependency graph on every build, even when nothing changed.

I've used the other major meta frameworks (remix and tanstack). I don't think there is a way to improve the speed of building the app in those ecosystems. Happy to be proven wrong.