Because even the "golden rule" leaves a lot of wiggle room - for example, should marital infidelity (or x) be a dealbreaker for electing someone? Maybe, but looking at hypocrisy (and thereby actions) might provide better insight as to what those groups actually believe is the "correct" standard?

> The closest we have to a "neutral rule" is "The Golden Rule"

Mostly agreed, but a surprisingly large percentage of the population seems truly unable to feel empathy for others (and can even show glee at others being negatively impacted by something) unless it personally affects them (a prerequisite for "Would you like it if I did this to you?" to work?). They can be gung-ho about laws that negatively affect others but upset if it affects them. For example, the countless examples of "Community members vote for politician promising illegal immigrant deportation, then upset/disagree that long-time beloved community member deported", or "Voter voted for politician promising criminal-record-immigrant deportation, then upset/disagrees that partner with past criminal record deported".

Just because something has a long history (secular or otherwise) doesn't give it any merit by itself.

If we need some standard to judge against what is morally acceptable in the case I presented (no this doesn't apply if you like to hurt yourself and others) then you are back to defining a standard which cannot inherently be neutral.