We can't create an algorithm determining whether a computer program halts or not, but we can write one that checks whether it conforms to natural language specifications much more easily? That makes no sense. There's no exception to the halting problem regarding "human comprehensible" computer programs.
replyThey're saying most useful programs don't fall in the complete / correct divide. You can get a lot done while restricting yourself to provable programs
replyRice's theorem says that we can't draw a partition between two sets of programs, based on their semantic properties. It says nothing about drawing a partition slightly to one side of the desirable partition, misclassifying some tricksy cases, but correctly classifying all the programs we care about.
replyI will give you a class of programs humans wrote and they want improved: LLMs.
replyThose were written by humans, and don't involve unsolved mathematics.
Is your claim tht you just need to solve comprehensibility of LLMs?
Figuring out epistemology and cognition to have a chance to reason about the outputs of a LLM seems to me way harder that traditional attempts to reason directly about algorithms.