Yes. When you don't have kids you can always quit the game/rat race one way or another if it gets too much. With kids - it is different. Kids are the only burden you can't easily shrug off in this life.
I assume you grew up in different circles than me. I have seen countless kids shrugged off growing up poor and around drug addiction. I have heard the rich shrug them off to boarding school sometimes.
I have seen of course. But the point is - if you abandon everything in life right now while childless you are not a failure as human being. It is your right to say fuck it all at any moment. If you abandon your kids you are.