Whether or not you'll be able to progress in belts/ranks is different but who cares. Wearing a white belt forever is probably better for character development. If your goal is the belt just go to the store and buy a belt.
There's also archery and I'm sure there are many other options for no-contact martial arts training. If fencing works for you then there's also Kendo. There's Iai-do which is just about drawing the sword. Lots of options in theory.