x86 servers are like that. PC/AT memory map have the range for VRAM, but RS232C is optional, so you can't just have the RS232C and not VGA on an x86 as its debug/recovery interface.

Platform designers can make up their own memory maps, so non-x86 ones can just have UART I/O buffer and hardware interface wired up in whatever address you want and forgo VGA.

I have a 32bit SPARC machine on a shelf I got from a junk hardware shop, it has a DB25 for console. It's ok.