It is easy to reduce people to caricatures when we don't know them. Since we aren't at a happy hour regaling each other over the craziness of lives lived, I'll be brief: there have been periods of life where this assertion wasn't true.
Communities built themselves around food production and child rearing 2000-8000 years ago as they nominally do today. I think you may be overstating the disparity in experiencing the negatives (drought, disaster, and war) and ignoring that these conditions also happen today and sometimes also don't get resolved -- people die, starve, drink poisoned water, and so forth.
Give me an idyllic migration through bountiful pastures 20,000 years ago any day. Many hunter-gatherers work less than 40 hours a week, less than a worker in our modern society, who did not get laid off in past years as so many on this board were.
The original affluent society as it has been called.
Every hunt was fight for life and survival, any injury could be fatal and treatment was practically non existent, broken foot was almost sute death sentence. Winter (and each one was harsh) was barely survived in bitter 24/7 cold and a lot of starvation outside of tropical regions.
You have no actual clue nor imagination with your first world whining about toils of office work which you can easily abandon and not be dead with your family within 2 weeks. You can definitely do better than that.