I would be interested to know what the actual rate of sperm count decline is, controlling for age and such.
Edit: This is good reading, I think, though dense: https://pmc.ncbi.nlm.nih.gov/articles/PMC6455044/
Rates are down quite a bit in the last 50 years, about half. But extrapolating to zero in 50 more years is a stretch. Average is something like 40-50 million/mL, which is above the 15 million threshold considered normal, but does mean an increasing number of people are falling below that threshold. So yes there is going to be an impact, extinction seems highly unlikely however.