they're still bounded by training data and how it relates to the real world.

Yes and no. They can extrapolate and build upon the training data, as was the case with the last dozens of math proofs