Not a natural language, but they are certainly a language as in a symbolic representation of information.
A sentence is a finite sequence of symbols drawn from an alphabet.
In this sense, mathematical operation tables are absolutely a language. As are natural languages.
A language is a structured system of communication used to express arbitrary ideas between multiple parties. Math operation tables do not, and cannot, do that on their own.
That distinction matters here because we are talking about what properties the model is expected to learn. English and operation tables are fundamentally different objects, so it is not surprising that a model learns different kinds of structure from them.