Yeah, it glosses over a gigantic capital expenditure. It's sort of like saying that an open source modern CPU architecture allows you to build your own CPU "for free" (provided that you own and operate a fab).

True. But there are other meanings of "free". I.e. nobody can say "from now on you no longer have access to model X because you're an asshole"

Hardware outdated in 12 months is FUD. What that would mean in practice would be either affordable consumer GPUs with > 32GB of VRAM, which doesn't look like it's going to happen, or unified memory systems with much higher bandwidth. That also seems unlikely.

You're better off setting a budget and buying the best machine you can afford in that range, or picking a VRAM target and accepting the class of models you can run on it. Those models will almost certainly improve over time and your skills will adapt to the limitations. Hardware is so valuable right now that it's not even likely to be a significant loss if you had to sell.

Right now I think 24 GB is probably the best bang for your buck (used 3090), because you also get a high end gaming/gpgpu device which is nice anyway. 32 GB you can do with AMD or Intel, but NVIDIA is megabucks and at this point you're really paying for RAM. Unfortunately the ship has sailed on "reasonably" priced RTX 6000s, which at one point were about $7k and are being listed at $10k++.