This idea is one I floated for my agent framework; it can get a little complicated if the model needs to reason about the redacted data. If not, excellent.

The other half of this equation is correctly marking PII/etc. This is a problem I'm relatively familiar with, at least as far as brute-forcing from raw files. I'd be curious to hear about how you managed this. Or is that something that AWS handles for you?