Would xcodebuild work in this context? Presumably I'd watch a log (or have an agent) and add permissions until it works?
Yes, Safehouse should work for xcodebuild workloads in the way you described - try to run it, watch for failures, extend the profile, try again. Your agent can do this in a loop by itself - just feed it the repo as there are many integrations that are not enabled by default that will help it.