That's not how patching a rom works. You directly replace data and the formats that let you undo have to put a copy of the original data in the patch file. So yes it is a risk.
Slightly silly idea to solve this: Hash all non-patched areas of the file to produce a key. Use the key to encrypt your XOR value. Thus the key can only be derived by someone who has either the original file, or the patched version.