I think the ideal way for these LLMs to work will be using AST-level changes instead of "let me edit this file".
replygrit.io was working on this years ago, not sure if they are still alive/around, but I liked their approach (just had a very buggy transformer/language).
>and most of the tokens are just echoing back the old code with minimal changes
replyI thought coding harnesses provided tools to apply diffs so the LLM didn't have to echo back the entire file?
They can, but this reduces the quality. The LLM has a harder time picking the first edit, and then all subsequent work is influenced by that one edit. Like first creating an unnecessary auxiliary type, and then being stuck modifying the rest of the code to work with it.
replySo, in practice, many tools still work on the file level.