They really shouldn't.

Does nobody remember why fizzbuzz was a thing? People who talked a big game while having no actual competence or understanding of the subject matter?

Nah. Separate things are separate until proven otherwise.

So much bullshit appears in the form "X is Y" where X and Y are related but distinguishable phenomena. If you really try to pretend that every such phrase deserves serious consideration just because it feels plausible to someone, you drown in nonsense immediately. (Of course what you actually do is only grant that consideration to ideas that feel plausible to you personally, but it's obvious why that's not a good principle, right?) So we have to make all of them justify their existence.

Anyway:

> So I’ll be handwavey here and say that if “actual understanding” is to an LLM what an LLM is to a bash script

No. None of this. Pretty sure this is just an incoherent analogy.

> so there’s a mechanism there that doesn’t just follow hardcoded paths, it takes new data and processes it in novel but human like ways...

You're putting your conclusion in the premise, right there in the open.

> ...then the author is right, in my opinion.

This still doesn't follow from your handwaved premises as far as I can tell.

Indeed. C.f philosophical zombies and the Chinese room problem.

The idea that some “actual understanding” (or more precisely, the outward appearance of it) needs a mechanism that includes free will is a bold claim.