LLMs don't actually semantically parse the error messages. They will generate the most likely sequence resulting from the error message based on their training data, so you're back to the training data argument.
replyThey process those error messages in the same way that they process your instructions about what code to generate. It is just more commands.
replyPerhaps the training data about what compiler diagnostics mean is particularly semantically rich training data.
replyOf course they do, error messages get tokenized and put into the context window just like anything else. This isn't a Markov chain.
replyExcept the presence of errors, mistakes, contradictions, and doubling-back causes LLMs to have substantially worse output, especially without dedicated sub-agents who have been instructed about that deficiency and know to process that kind of crap into better prompts to insert into a different LLM with pristine, error-free context. Without hard numbers we're both just pissing into the wind, but it's entirely plausible that the higher rate of errors matters more than the fact that those errors are more ergonomic. Anecdotally, my LLM work is a _lot_ more productive when I have it draft the thing in Python and translate it into Rust since it wastes so much time on the tiniest of syntactic mistakes.
reply