While I agree that for some people games with a time limit are not fun, I don't think the challenges and difficulty should be classified as "purposeless" and "dumb". For many the challenge/difficulty IS the fun part, and they serve a genuine purpose. If you don't like that, then play a different game, but that doesn't mean the game you don't like is useless.
Tangentially, I have noticed some of the most well-balanced difficult games I have ever played were the ones with very granular difficulty settings. Examples include CrossCode and Celeste. Crypt of the Necrodancer too, though the customisation there feels like it crosses into too granular. In each I changed the difficulty settings exactly once, for optional challenges, and it made the games way more enjoyable.