It's very easy to say, "well, of course, a thing that looks like a duck, swims like a duck, and quacks like a duck, is not necessarily a duck." But when you're presented with something indistinguishable from a duck in every way, how do you determine whether it's a duck? You can't just say "well I know it's not a duck". It's dodging the question.
replyWell. AI doesn't walk or quack like a duck.
replyAsk it to count first two hundred numbers in reverse while skipping every third number and check if they are in sequence.
Check the car wash examples on YouTube.
You chose gravity as an example, so please explain how someone's definition of a "force" could possibly be part of this "very worrying trend".
replyAnd this logic flow only proves that no AI is a human intelligence. It doesn't disprove the intelligence part.
Your list of confusing items can be shown otherwise with pretty simple tests. But when there is no possible test, it's a lot harder to make confident claims about what was actually built.
Would you claim that relativity disproves aether theory? Because it doesn't really. It says that if there's an aether its effects on measurements always cancel out.