While people do often use pis to do things you could do with an esp32, an ESP32 cannot do the majority of things a Pi can do.
Personally I use ESP32s for GPIO to control things around the house and have one Raspberry Pi acting as a hub.
In some of the tinkering applications, RPis are shoehorned into the very operarional model that is followed by ESP32s.
ESP32s may be harder to debug but they allow us to remove a big chunk of the complexity that lies between our code and what we want the device to do.
I think your criticism is more in line with "they are not exactly the same as RPis". And that is fine.
Microcenter sold it for $5, limited to 1 per customer. Additional ones were $15 each.
A pi zero is to play with, an esp32 is so tiny, that it is more of a build for a single thing, and then set it and forget it doing that single thing.
You cant really stack more and more cool stuff onto it and keep building on it.