Once that was done the flood gates opened and many others broke it in the following months. This is the “central governor” theory in endurance sports.
Progress had been steady for decades, but was interrupted by WW2.
There was one other person who did it a sub 4 minute a month and a half after him, then 3 more people the next year.
It was more down to improvement in training (Bannister was doing interval training, which was a new idea at the time).
We are assuming the old record is "the best a human can do because one person did it best" or some form of that.
There are likely hundreds or thousands of people alive right now who could break this record given the same lifestyle and training.
IMO, better training counts as "raw performance". I think that's more interesting than somebody happening to be born with a genetic advantage.