It probably would. LPDDR5 memory bandwidth on M-series Macs is several multiples of the memory bandwidth available on DDR3 (depending on the exact chip, Pro, Max, etc... 4x to beyond 10x.) The whole system would be much, much slower.
Yeah, sure but... if it's real fast, lets connect SSDs to it and have real fast swap. Whatever it is will be limited to the bus speed. And that's gonna be silly compared to RAM speed.