Actually - do they do this in LLM benchmarks? As a measure of overconfidence/confabulation? Seems immediately applicable.
reply"incomplete information" is a standard concept in word problem curriculum. But usually it's explicitly an option in the test, as a fairness to the student.
replyMaking mistakes in lecture is a standard technique used by good teachers, to promote active listening and critical thinking.
I didn't see that in the document. What page is it on?
replyI think they mean at the bottom of p216 (pdf page 4), where he says he doesn't know, r+s=80 but there isn't enough information to solve for r and s.
replyThere's two questions that are intended to be wrong (probably to test confidence). One with insufficient information and where the question itself implies falsehoods.
replyThe questions are on page 215 (3/26) and Tao's answers are on the next page.
reply