DRAM prices at mid-2025 rates were ~$2.5/GB for DDR5, and ~$1.5/GB for DDR4. "Hundreds of gigs" of RAM used to be under $500. 128GB of cheapest RAM used to be like $200. It seemed to go over heads for a lot of people that you could get hypothetical future machines on CS/CE textbooks were attainable for that little, for some reason - there seemed to be some fixation on the idea that 16GB is all you need.

You don't have to have a server, workstation motherboards support lots of memory channels.

I was lucky to buy a lot of RAM before prices skyrocketed. I knew I wanted to play with this stuff, so I spent what felt like a lot of money at the time to buy 8x96GB DDR5-6400 RDIMMs. Now the same RAM costs at least 6x more.

As soon as Llama came out I had a realization what was coming and went all-in on hardware with the assumption open source would catch up with GPT4. Surprise, it did, we now have small models that absolutely crush GPT4 in performance.

It wasn’t that absurdly expensive for a hobby, I bought 64GB DDR4 ECC sticks between $70-$100 on eBay before everything took off. Now everyone is in here debating if open source is 1 month or 3 months behind SOTA. The future is obviously local.

I got a 2U rackmount with 192Gi DDR4 for $1.1k USD in 2023. Around 1.5 yrs ago, server RAM could be had pretty cheap--especially slower LRDIMMs (I wanna say 512Gi DDR4 was <$500 USD). I checked a couple old ServeTheHome threads and seeing maybe around $50/32GB RDIMM although thought it was cheaper than that for a little while

RAM wasn't expensive even a year ago. I maxed out a used Dell Precision T5610 with 128 GB DDR3 for $250 in 2021.