Readability is certainly a limit. JSON and XML are unreadable in their usual single-line transport form, and often even when pretty-printed. XML signatures break upon reformatting, so you also can’t just do it blindly.

Part of the lowest common denominator are the (printable) ASCII characters. If you ever opened a text file mostly consisting of a script you’re not familiar with, it might as well have been binary. Add to that right-to-left languages where you can’t even be sure which element follows which without knowing the scripts.

It’s “good enough” for many purposes, but it’s important to keep in mind the limitations.

I think the real limit of plain text is pretty obvious: you cannot embed pictures in it.

It’s like SMS vs MMS or modern chat. With pure text, you can at best add a link to a picture (which could get rotten or inaccessible for other reasons), but you cannot directly graphical content.