Yes, serving an MP4 file directly into a <video> tag is the simplest possible thing you can do that works. With one important caveat: you need to move the "MOOV" metadata to the front of the file. There are various utilities for doing that.