This is really only true insofar as they are designed such that they output language tokens and their intermediate "chain of thought" is also done in language tokens, but now that's really only done for auditability. There are much more abstract layers before the model output layer of a single token prediciton and they could be designed to to "think" using these more abstract layers as intermediates instead like how many people think in concepts rather than words. There are even some research models on this (see Next Latent Models)[1]. Humans need to communicate in language because we don't have telepathic abilities but these models need not have such a constraint.

[1] https://arxiv.org/abs/2511.05963

It does not matter whether LLMs do any thinking, or "thinking." The only thing that matters is whether they produce similar results to human thinking, even if it's through a different process (or no process at all). When a company decides to deploy an AI-powered customer service representative, it is not because the AI bot goes through the same thinking process as a human when answering a customer's request.