edit: Not code, just convention.
The more practical reason to mount ground down is that wall warts with ground pins or polarized prongs nearly universally arrange them so that they're hanging down when inserted into a ground-down plug. If the plug's flipped, the wall wart's upside down and its weight is trying to lever it out of the wall.
TBH, in the house I mount them ground down, but under cabinets or in the garage/shop or etc I mount it ground up.