They process those error messages in the same way that they process your instructions about what code to generate. It is just more commands.

Perhaps the training data about what compiler diagnostics mean is particularly semantically rich training data.

Of course they do, error messages get tokenized and put into the context window just like anything else. This isn't a Markov chain.