upvote
Yes, this is often obscured when working with code because it's relatively easy to judge its correctness. When using Claude to write documents, describe a system or organize requirements it becomes clear very quickly that Claude doesn't really understand what it is writing, it just puts probable things close together.

Doesn't mean Claude isn't super useful though.

reply
I think the problem here is the coding agent that we are working with are not familiar with the degree of knowledge we have around the system

It's like throw a lot of concepts at once that were just created 5mins earlier but the agent doesn't care that the human supervisor isn't aware about them and how they function, so it doesn't know what simple explanation in relative terms should be provided next

I think this is also implied in the article in the ending paragraphs

reply
There are skills which try to force Claude Code to use Feynman technique like [1] and many others

[1] https://github.com/TheJavaGuy/tjg-claude-marketplace/tree/ma...

reply
Well surely claude struggles here because it doesn't '''understand''' anything.
reply
When making such strong claims that could easily devolve into semantics, it would be valuable to explain what you understand to be the definition of the semantic fulcrum of the claim.

In this case, that semantic fulcrum is what it means to '''understand''' something. Because the question of whether Claude understands things hinges much more on the criteria for understanding, than it does on our belief on the capabilities that Claude has.

reply
Sure, I was mostly saying that to be a bit adversarial. I will say though that claude has had much less RL in certain areas, such that an explanation of e.g. why FTL travel violated causality is mostly remixing answers that are in the training data rather than a serious effort to facilitate understanding.
reply