Although thinking about it more, the fact that address indexing is now from top-to-bottom is actually consistent with how I imagine indexing for Nx1 array (or equivalently, how you index matrices in math).
Thinking about it more, I don't think there is any real convention, even diagrams are split. You are going to have to learn to "flip" things either way, the same way matrix indexing differs from cartesian plane indexing. Same way different graphical systems have different conventions for where the origin is. People colloquially say things like "high mem", and you'll have to translate this to your mental model.
It's why I suggested a horizontal, left-to-right visualization. I think everyone would agree that lower addresses on the left, higher addresses on right.
Also could you elaborate more on what you mean by debuggers showing it the opposite direction? If you do `info proc mappings` in gdb it is also lower addresses at top. This might be debugger specific though.