I think Feynman might have been referring to how spin was not understood (even by him), and magnetism is a result of a bunch of little spins adding up. I mean—even today—there are not really explanations that are correct and also understandable by a freshman. There's the belt analogy but, well, it doesn't really explain the physics.
That's not how I read it. Feynman started with the story of a girl going to the hospital because she broke something after slipping on ice. His point was that you can only explain things in terms that you simply accept without further explanation, otherwise you'd keep asking "why". Then he muses about imagining little rubber bands between magnets, but those only have their property in the end because of electromagnetism, so it's a circular explanation and thus unsatisfactory. To him, the magnetic field and its properties simply are, there is nothing underneath it.
Didn't Feynman mean "layman terms" when he muses how he doesn't quite know how to begin to explain magnetism to his interviewer (but goes about doing so in 'simple' terms, anyway)?