If you're going that route, add CSP headers on HTTP level to disable scripting, and/or host the SVG on a separate domain that has nothing valuable, or use data: URLs.
But also so that setting up a CSS transform: scale(10000) can't take over the entire viewport, it'd be constrained to an iframe-like boundary (exactly like an <img>) but still remain as an inline SVG, sort of like an <iframe srcdoc>. So scripts on the parent/host HTML document can still manipulate it like the rest of the DOM, but the inner <svg> elements are all "inert" for want of a better word.
Actually I don't know off the top of my head what happens with an SVG file inside of a <img> when it references external images (either cross-domain or not.) I know scripts and animations get disabled, so I'd take a guess and say some CSS gets blocked too.
Again I've not really thought terribly hard about it, or if it's actually useful at all, and I'm betting it'd be filled with even more foot-guns than there are right now. I'm just thinking out loud.