The 48k Spectrum had a 1-bit "framebuffer" with colours allocated to 8x8 character tiles. Most consoles of the time were entirely tile/sprite based, so you never had a framebuffer in RAM at all.

I think it's a valid view that (a) we have way more resources and (b) sometimes they are badly used in ways that results in systems being perceptibly slower than the C64 sometimes, when measured in raw latency between user input and interaction response. Usually because of some crippling system bottleneck that everything is forced through.