I've had good results getting AI to do create both an app guide, and do naive UAT testing - it sometimes gets lost in the weeds, and sometimes hyper focuses on things that aren't important, but it also clears the deck of knuckleheaded things that would waste human UAT testers time. I run it any time major features goes in, and it regularly clears up things well.

I feel like your suggestion would be a nice organic extension of that same practice (at the cost of more infrastructure/tokens)

yes