upvote
Peano Arithmetic has lesser proof-theoretic strength than ZF. Iirc, if you remove the axiom of infinity from ZF, and then maybe add in its negation or something like that (I don’t remember the details), you get something with the same proof theoretic strength as PA (or a mild strengthening of it?).

In particular, the hereditarily finite sets are in correspondence with the natural numbers.

There’s a fair bit of mathematics that relies on the axiom of infinity. Uh. And a lot also uses the axiom of choice (or something equivalent), though ZFC doesn’t have a greater consistency strength than ZF (both can prove that if either is consistent, then both are).

I think it is fair to say that one’s choice of foundations does matter at least a little bit, though typically not all that much.

That’s not to say that it is always necessary to state what foundations one is using. One can understand a work to be done in any foundations in which it works. Though sometimes it is important to specify.

reply
That's sort of the argument for why we have so much faith in set theory though, and things provable under it. It's not a system with some special powers that let us prove more things, it's a system that only lets us prove things that are intuitively true under "any" proof system (not actually any of course - but as you say a variety of equivalent foundations).

Reducing the correctness of the lean code here to set theory is then also reducing it to everything you can reduce the correctness of set theory to.

reply