This farmer needs a tote.

It did get it wrong but it also got a lot farther than much more recent, but worse models like 6.7GB on disk size ternary bonsai. It at least knows it's from Yiddish. The "schlemp" appears to be a total hallucination or it's confusing it with schlep, which is not related to schlong. One of the reasons why I said it "mostly" passes the test. Something much larger on the size of qwen 3.5 122B, deepseek v4 flash or similar that runs in 120GB to 190GB of RAM in my experience will answer perfectly unless it has been ruined by something like Q2 quantization.

There isn't SchlongBench(TM) yet, it's a specific question I've been asking of differently sized models as a randomly chosen gauge of how much less commonly used knowledge is perma-baked into it. In this case a question about a specific yiddish origin slang term. Small/bad models don't know it's from middle high german or Yiddish and get its origin and meaning totally wrong (or it runs into model censorship related to slang related to the male anatomy).

It's also a question I have found will cause models that don't know what it is to go off quickly in a direction of hallucination trying to explain it, so the hallucination is evident very quickly starting from the first ever prompt issued with 0 context fill. Example: I had a model write four detailed supposedly-accurate sounding, grammatically correct paragraphs saying its origin is from AAVE (African American Vernacular English), which it most certainly is not

You could do the same by picking any topic that is very rarely discussed in conversation, some esoteric and narrow piece of knowledge and asking the model about it.