It's a pretty normal thing to do for small LCD screens. Linux has had SPI framebuffer support via fbtft subsystem (in staging tree now, previously was out of tree) for well over a decade. It works quite well.