I think what you suggest is like a local second order approximation, that can help. But, I think that the real problem is a global one, is about architectural taste, how the many local pieces interact and their friction. Currently that demands human expertize.

> I think it's taught to look at source code through a peephole for the sake of context preservation

Yes, to a (real) fault. Less than one in fifty times it ignores an instruction or piece of data in a file has it seen the instruction or data before ignoring it. The other times, it's done this sampling nonsense.

Results are night and day using the 1M token models and reading the full files.