"download everything" means that you don't have to do a full or partial clone of the repo to make a change. You can imagine agents running in the cloud need to spin up and access the repo for every task, this can be slow for large repos (also small repos with large files). Also locally, worktrees can be a pain to manage with conflicts, confusing branching, and you can't check out the same branch multiple times. But I do agree that we're probably still pretty early on in the agentic adoption that many users using agents in git-like ways will not see much performance improvement.

I could, I hit it in some situations. But I'm running some fair size repos where I do multi-worktree work.

That said, it hasn't been enough of an issue for me to want to fix.