My point is precisely that I was often taught physics as if it was mathematics, where there is in fact a profound ontological difference between the two.

Also, physics (the discipline) is also a construction by and for humans.