I think it would be "fly vfr" but the main issue there is the wind direction, not the altimeter setting. In daytime you can read the wind sock while airborne but not possible at night.