I've been doing the same thing lately and I definitely feel like stubbing out the high level architecture at the beginning makes a difference. The codebase I'm in now has very particular ways of doing things and claude doesn't always pick that up.

Style can be as important as substance.

I still do a lot of back and forth about the plan - have it written to a file. Read through the file, make changes by hand and have claude read my changes and on and on. But starting with the basic architecture there's less ambiguity.