There are many political opinions that I strongly believe in that I am comfortable disagreeing with people on. I believe everyone has a right to health care, and that society should guarantee basic necessities for everyone. I even feel that belief is a morality based belief. However, I can accept people disagreeing with me, and can accept that there are some strong arguments against my belief, and that good people can disagree with my position.
On the other hand, if someone believes that certain races should not have the same rights, or that women should be given less agency than men, I will not entertain that argument or accept that it is just a political dispute. That is a fundamental moral issue, and is beyond JUST politics.
As far as your particular question goes, I don't agree that believing that all races should have the same rights is inherently in conflict with the idea of affirmative action. In most implementations, there are no rights that are denied to anyone when affirmative action policies are implemented. The entire point and purpose is to counteract existing norms, institutions, and system structures that are actively denying rights to citizens in particular groups/races.
For example, take the original affirmative action order (from which the phrase was coined) signed by JFK in 1961. The text stated, "take affirmative action to ensure that applicants are employed, and employees are treated [fairly] during employment, without regard to their race, creed, color, or national origin"
What rights are being denied if that is followed? The idea is that it is clear through observation that the criteria that was being used before was preferential to white Christian men, so they were instructed to proactively address that unfairness by changing their hiring process to attempt to eliminate those biases. How is that in any way denying rights to any group?