Key parts of the abstract:

> This raises a question the programming-language community has not previously had reason to ask: should error-message detail be calibrated differently for AI agents than for humans?

> We investigate this question through a controlled experiment using Shplait, an ML-style statically typed language. We construct a suite of programs containing a single deliberate type error each, and measure how often an AI agent repairs them under ablation: a detailed error context using the unification stack; a proximate error location; a minimal type error; and a dynamic (test suite) error only. An automated oracle uses a test suite to classify each repair attempt as a type error, semantically incorrect, or semantically correct.

> We find concrete evidence that more detailed error messages generally improve an agent's ability to fix type errors. We also find that the presence of a type system appears to help more than only test suite failure reports.

That actually would be pretty exciting!

But, wouldn't AI be biased toward more popular languages, since those will by nature of their popularity provide more sources for training material?

Even still, the AI could demonstrate which algorithms or maybe patterns and techniques are more productive, in the context of the popular languages.