And sure, you can find a group of like minded people and go fully off grid, and live that life of "leisure". But your idea of leisure better be farming all day, being hungry with bland food all winter, and a gash on your toe being life threatening.
Usually when people conceptualize stuff like this, they do it on a personal level without consideration for what society on a whole would look like if everyone did it. If you keep digging, you find that 99% of people actually just want benefits of others work without working themselves. What a revelation!