Also past LEO you don't really have issues with fallout.
So the only real hurdle is securing the supply of couple thousand nuclear bombs for the trip.
But if you told people in 1903 that you can build a plane that will get 850 people 14000 km but you need 250 tons of kerosene to fuel it, they would also think you are insane (e.g. an Airbus A380).
But over time, even with Orion, people will just get used to it. :)
If we consider the low end your time estimates, we could theoretically have received data from Alpha Centauri by now.