I've had the same thought on self hosting for the past 2 years. Eg i could invest $10k into hardware or spend those on credits. I chose credits, because by the time i spend 10k i suspect the market will have dramatically shifted in requirements, compute, etc.
I agree, at least for inference credits. But anecdotal counterpoint: about ten years ago I put together a system using about $1600 worth of new and used components. Including electricity that TCO has been something like $14/month. I'd be hard pressed to find 128GB of RAM, 12 cores of x86_84, and 40 TiB (raw) worth of disk, and unlimited ingress/egress traffic for rent at that rate. That's what, $0.02 an hour, rounding up?
Yes, a hyperscaler's rent-a-computer would get faster and be more energy efficient. Possibly more carbon efficient over it's lifetime, although I have no way to quantify that. But in raw dollars it would have costed me absurdly more to rent. On the other hand, my machine has sat cold and unused for multiple months when I didn't need it to be running, and I don't consume all of it's resources around the clock. So maybe I could have gotten away with a lesser amount of local compute in a Raspberry Pi or similar, and an equally small cloud box for $5 a month or something. I dunno. I don't even own a laptop or desktop, and try to use 'cloud' services whenever practical. (Edit: always giving priority to privacy in those circumstances; only data I don't care about goes up.)
There are tradeoffs. I don't use my local server for LLM inference, it's far to inefficient for that; so I'll happily pay OpenRouter per token as needed. So I do think that hosted compute for rent does have a purpose. I guess all that's to say that I don't see it in black and white; there are times to invest in local hw and times to ride on the optimized bleeding edge of what other companies are willing to host in big datacenters.
This turned into a bit more of a ramble than I intended.
I just also think that plenty of people buy clusters of Mac Studios anyway.