That depends. It can still affect genetic fitness if it affects an individual's ability to confer benefits on their descendants. Of note: most of the most wealthy and influential people in our society are beyond their reproductive years (not technically true for men, but mostly true in practice).