upvote
"The reversal curse", it rarely shows up in practice but you found a case when it did.

The "knowledge landscape" an LLM uses is "directional". It's easy to reach "a quirky music band from Michigan known for colored ties" when you stand at "Tally Hall". But if you stand at "a quirky music band from Michigan known for colored ties", it's harder to reach "Tally Hall" from there. For the "latent knowledge graph" an LLM uses, A->B doesn't cause B->A.

In practice, any "common" facts will have enough "traversal" in both directions that this directional biasing isn't apparent. So it only shows up on this kind of more obscure knowledge.

reply
If this kind of thing holds true for humans we now may understand synesthesia, perhaps we find ourselves with a large enough study pool to map out clever tricks to influence each other.
reply
It certainly holds true for humans. The brain stores relational information in a sequential pattern that is not automatically reversible. One of the best examples is the alphabet. Everyone learns it in school, so the pattern A->B->C->... is trivial to recite for most people. Now, if I gave you a random letter to start with and asked you to to recite the remaining letters until Z, you'll probably find it is still pretty easy. But if I asked you to cite the letters backwards to A, most people would suddenly struggle with this task because they never learned or used the alphabet that way in school. You need to train specifically to link this kind of information backwards in your brain.
reply
Is the alphabet really the same though? I don't feel like I recall it one letter at a time, as individual facts linking A to B, then B to C, etc, but more as a sound or a phrase. Not unlike recalling a melody. It just seems very different from figuring out what band someone is describing.
reply
Ask someone to name all fifty states, see how far they get. Then start naming places and ask if they're a US state or not. One of these tasks is far more likely to get a 100% hit rate.
reply
Not that I'm too sure about how this applies (or whether it applies at all) to other people as well, but for me personally it's easier to recall information about the band if I'm being told the band's name instead of being told "well, they are known for coloured ties". So, there is certainly some kind of effect described in action.

Now, about alphabet: again, I think it's only me, but when I try to recall it backwards, I can't do that easily. I mean, I can recall it backwards, but I need more time to do that. It's harder. I'm not sure if it's because A links to B, B to C, then C to D and not backwards, or maybe just because in school you learn alphabet from A to Z and not from Z to a - so you're kind of trained to recall it A-->Z way - but it's certainly harder for me.

At the end of the day, though, I think that everyone thinks differently. Everyone is having different internal representations for concepts (such as alphabet), so it’s not surprising that this effect may work differently for different people, or not work at all.

reply
After writing the comment I thought about it some more, and realized when asking myself the question ”what letter comes after R?”, I didn’t immediately know the answer, but I heard ”MNOPQRS” in my head which gave me the answer. So I feel like I know it because I know the rhythm and sound of saying those letters in sequence.
reply
deleted
reply
I think what you're saying might be a stretch. I strongly believe the brain holds some information in a sequential pattern -- not necessarily all of it or even the majority of it. There's a book called Moonwalking with Einstein that digs into this precise fact -- you can remember sequences much much better if you associate a visual image with each item in the sequence. Sequential association covers a lot of human knowledge but certainly not the spatial aspects. I think it's a big reason why I think LLM's might not be as smart as we think they are. The complex spatial representations are only encoded implicitly via projection on to textual descriptions.
reply
But it does cover the state described in the top comment. A<-B is never going to be easily retrieved if you only experienced A->B during training, regardless if you are a human neural network or an artificial one. Also, you need to define "spatial" better. This is about logic after all and not geometry. Or topology? It's unclear which context you refer to. It's certainly not the topic of this thread.
reply
Its very true for humans as well. For example, if you ask someone to "name 100 historical figures", they will have a very difficult time. But there are of course hundreds of historical figures people woukd recognize. People who manage to do this still tend to fall into sequences so, i.e., naming all the presidents of America or emperors of Rome. But the broader the category, the more difficult it is to come up with examples.
reply
What's this got to do with synesthesia?
reply
deleted
reply
[dead]
reply
Recall isn't naturally bidirectional, even for humans. If you are learning vocabulary in a new language, it's common advice to practice both target > source and source > target. Doing only one-way often makes you much better recalling that single direction than both.
reply
I would need further convincing that humans do not naturally tend towards bidirectional recall.

Perhaps I’m just on alert anytime I see an LLM-ism that’s met with a claim that the same or similar phenomena holds true in humans as well.

reply
'Naturally' might not be the best word? Maybe 'Necessarily' would be better?

Regardless, it's something that happens in people. Have you not or seen someone else struggle to recall a specific fact or memory until phrased or induced in a certain way?

You probably could also say LLMs 'tend towards bidirectional recall' over the course of training as things that ought to be recalled both ways are reinforced to do so. In the above example, you will also eventually learn both ways with enough exposure even without explicit practice.

reply
If you can touch type try this: Go through they keys once alphabetical 'abcd...' recalling the movement from the letter. Then do it the other way round and move your fingers in spatial sequence to the keys and try to recall the letter.

You will find the former much easier if you did not by chance also memorize the keyboard layout for some reason.

reply
This is very obvious any time you try to create exhaustive lists of something.

For example, for every country in the world, I would recognize it and say, yeah, thats a country.

But if I had to write all ~200 countries into a list, I would probably miss quite a few.

Or, if you gave me names of all US presidents, I would for each of them go, oh yeah, thats a president. But ask me "name all the presidents", I wouldnt get further than 10.

reply
This is something that anyone learning a foreign language could tell you is very much a thing.

Understanding a word when you hear it, is frequently much easier than remembering the same word when trying to speak/write the language.

reply
Do you mean that you don't believe that humans learning a language have the problem the parent described? Because I do, and everyone I've ever met while learning does as well.

Do you mean that you don't believe the problem exists in general, because here's another example: if you give a song title, I can easily hum the opening. If you give me the opening, I cannot reliably name the song.

reply
[dead]
reply
What about the fact that "What is the weird band from the early 2000s in Michigan who wore colored ties?" could be a bar trivia question (challenging enough to recall to be fun), while "Who are Tally Hall?" could not
reply
Only because one is a short answer and the other is not
reply
Probably an instance of:

"The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"

https://arxiv.org/abs/2309.12288

reply
Really interesting paper, thanks for the share.

The point their making in that paper reminds me of this paper some people shared around work earlier this year, https://arxiv.org/pdf/2512.14982 (Prompt Repetition Improves Non-Reasoning LLMs)... I wonder how OPs question would fare (or the questions presented in the paper you posted) given double repetition.

reply
A relevant question for the paper you linked is what happens if instead of repeating the input prompt you repeat a filler character the same number of times.
reply
This is tested in the paper (see the "padding" ablation)
reply
Can you elaborate on that more? Why just a filler character?
reply
Because padding the output in such a manner (which keep in mind is immediately fed back as input) has been shown to increase model performance. It's one of the many reasons to question what "thinking" traces are really doing.

If you consider how the attention mechanism works then a very hand wavey intuition is that despite being entirely arbitrary additional tokens should still provide the opportunity for additional information processing.

reply
This is also something to be aware of when teaching people, too. I've seen advice for designing Anki-style flashcard decks that reminds people to create flashcards for both A->B and B->A.
reply
Can an LLM self-fine tune by generating reversals?
reply
I asked a local instance of Qwen3.5 122b: "American indie rock/pop band that formed in Ann Arbor, Michigan, in 2001 best known for their eclectic musical style, whimsical lyrics, and distinctive visual aesthetic involving matching colored ties and hats?"

Got this back "The band you are describing is Tally Hall.

Formed by friends at the University of Michigan, they became well-known in the mid-2000s internet era for their songs like "Good Day" and "Rooftops," as well as their signature look where each member wore a suit with a tie and fedora in a specific color (Red, Blue, Yellow, Green, and Black/White)."

Update: The gpt-oss-120b also got it correct on my prompt version. Update 1: Llama 3.1 70B gets it right as well. Update 2: Deepseek V4 Flash gets it right. Update 3: Glm 4.5 Air gets it right as well

Can't seem to get any smaller models to get it.

So seems it could be an issue of data points on which the model can latch onto, the more the merrier, as they say. When asked the "What was that weird band from michigan from the 2000s that wore coloured ties" was not able to get it right.

reply
There's a shockingly unfathomable level of data compression that these local LLMs are able to remember in an encyclopedic way, so many obscure little things.
reply
Is it? Compressed (text) Wikipedia is in ballpark of dozens of GBs.
reply
Dozens of GBs of compressed text is an unfathomable amount of information from a human POV.
reply
If I ask my kid (early elementary) what he learned last school year, I can watch him apparently genuinely trying to think of some examples and not being very successful. If I ask him something I know he learned he’ll gladly say “oh yeah!” and tell me about it.
reply
LLMs aren't humans, humans aren't LLMs. The fact that your kid can't recall what they learned without you reminding them is natural; the fact that the LLM can't recall something without a reprompt is a failure mode.
reply
And yet (almost?) all of the quirks people keep bringing up about LLMs are also quirks that human cognition share.

It's almost as if artificial neural nets share emergent behavior with natural neural nets.

Pop quiz, what's the difference between natural synthesized vanillin and artificially synthesized vanillin?

reply
dude, you could ask me, a grown adult, what I did last week and get a similar lack of recall
reply
https://claude.ai/share/2b0f85a2-e7b8-4f62-91a0-eca61bdeabec

Fable 5 on low gets the answer with web search turned off, one-shot!

reply
Ah, that big model smell.

Every time someone somewhere says "an LLM can't do this", the next generation of LLMs gains one more parameter. Until that LLM can, in fact, do this.

reply
So the model was updated in the 37 minutes since OP posted his comment?
reply
Just a funny observation. Every time someone proclaims "LLMs can't do X", a bigger, badder LLM that can in fact do X shows up shortly thereafter.

Clearly, Fable 5 didn't even have the decency to wait until the next model refresh cycle to show up. It was already sitting there waiting.

Either the capability gains in bigger, badder models are actually unrelated to "gotchas" being discovered, or LLMs are already acquiring Skynet levels of disrespect for cause and effect.

reply
Or just AI denialists like to say "LLMs can't do X" even though they can and have been doing it for the past few months or more. They only get called out once the current SOTA LLMs get so good at it, that any rando can trivially and reliably falsify the claim on the spot with whatever SOTA LLM surface they have handy.

Which I suspect is what happened here, given the trail of smaller / local models that successfully answer the question, too.

That said, "curse of reversability" is real, as much for LLMs as it is for people.

reply
I don't think it's solved this fundamental architectural problem by itself, it will have just squeezed the edge cases thinner. It keeps happening, people find a question it gets stupidly wrong, the vendors proclaim they've fixed it, then another one gets found.
reply
What does it say to the second question? I've found Claude is one of the worst models with regards to pop culture knowledge like this, even compared to the Chinese open ones. Just curious, not really relevant to the initial post but I don't pay for it so I only have access to Sonnet.

https://claude.ai/share/5e7e09b2-a75a-4024-b261-9a1a4e063a8b this is mostly hilariously wrong. wrong tie colors, they did not replace their bassist with a drummer, two completely made up albums, the rob cantor song it is thinking of is "shia labeouf", and a few fan behaviours i think it just made up

reply
"I've found Claude is one of the worst models with regards to pop culture knowledge like this, "

Is that bad?

I want to use models for coding and reasoning capabilities, not pop trivia knowledge they can get with web search.

reply
For most cases probably not. It's just something I like testing new models on sometimes, the pelican riding a bicycle benchmark probably isn't that useful either.
reply
But that is just testing for encoded knowledge, the pelican riding requires some reasoning capabilities, but lost its surely usefullness a while ago.
reply
Thank you for prompting me to try with my own obscure question, Fable was able to find something from a poor description. I've been looking manually and over many sessions with different models as they improve, none have been able to find what I was after. Fable 5 just one-shot the answer.
reply
Gemini 3.1 Flash-Lite with web search disabled through the API and zero thinking gets it just fine.
reply
fwiw o1, o3 and 4.1 also give the correct answer (without web search)
reply
This doesn't seem that weird to me. Talk to any human and you'll find that their ability to recall specific names and facts is very context-dependent. Phrasing a question in one way can make it hard to answer, while providing certain words or cues can instantly "jog" the memory.
reply
The LLM is a machine though, and the human is not. They don't think like us. Any similarities that you see are just coincidence, not purposeful design.
reply
I still don't see why it's surprising, though. Humans and machines are subject to the same laws of physics. While we wouldn't expect LLMs to think exactly like humans, we should expect some degree of convergent evolution. There are always tradeoffs between storage capacity, accuracy, latency, relevance, fluency, etc.

Expecting LLMs to be magically perfect information retrieval machines is never going to be realistic.

reply
Just today Google’s AI told me that 3**4 was 181 because I asked it for bases such that when raised to the 4th power had a 1’s digit in the leftmost place.

Yeah, LLMs aren’t designed for this kind of thing but it was really confident in its assertion… it picked the example too!

reply
Hate to break it to you but Gemini 3.1 Flash-lite with grounding disabled (through the API) gets this first try, even with thinking disabled.
reply
Yes and several other models that know about the answer don't... not sure what you're trying to break
reply
deleted
reply
I'm not sure which chatbots you used, but OpenAI's o3, o1 and 4.1 get it right first try (used in the Playground without web search or any other tools).
reply
I tried o3, 5.3 instant and 5.5 high and they all found it instantly with search disabled.
reply
deepseek v4 pro which doesnt have search feature could answer it
reply
I think most thinking models can do it to be fair, I think when I tried this it was all with non-thinking models/modes. Wasn't trying to make a point that LLMs can't do it or anything, just thought it was weird.
reply
P(i|j) != P(j|i)
reply
Not sure why this is weird when you really think about it. You're saying that a multiparameter constraint satisfaction problem is harder to solve, even when there's only a single solution (one point in the total space that satisfies all constraints), than just being given that solution/point. Well of course it is!
reply
[dead]
reply