> Isn't it possible to somehow wire this into the window manager? Wayland or whatever. Have it speak the native window lang rather than crunch the pixels? At least for the majority.

Not possible on wayland, maybe on X11 protocol?

Wayland only has pixels. It was designed to get rid of all the X11 cruft.

imagine, if you will, that we had a windowing system that's built on Postscript... lots of folks thought it was a super awesome idea, and built NeXTSTEP around it. https://en.wikipedia.org/wiki/Display_PostScript

or even one based on PDF like OSX: https://en.wikipedia.org/wiki/Quartz_2D