That doesn't work well with text documents which won't have any kind of magic number. So now you're doing some heuristics to determine is this text/plain, text/html, text/svg? You're pretty much just guessing at that point.
A good number of file formats out there are just Zip files with a particular structure. JAR files, docx - so relying on magic numbers doesn't really work for those, either.
Also to service a HEAD request you'd have to open the file and read a few bytes that you just discard.
If you just do it by extensions you don't need to read files at all or perform heuristics, and no ambiguity for what mimetypes to use for text documents, zip-based formats, etc.