To that end, if your goal is just to read more, there is no reason to worry about how substantial your books are. However, if you goal is to read more substantially, you should start by aiming a bit higher than where you are. Achieve that, then adjust target.
Progress, then, can come either in more volume of reading where you are; or in more substantive reading. Either are valid, to me.
To take this to the exercise. If your goal is to do a fast mile, agreed that just walking the dog is unlikely to help. If your goal is to be physically active, simple walks punch well above what people think they do.
Book people really hate to hear it, but literary fiction follows Sturgeon's law just as much as Sturgeon's own genre. 90% of it isn't worth reading, and that includes 90% of what's fashionable at any given time. You're better off reading books you enjoy than suffering through garbage that you're told you're a bad person if you don't read.
(Literary fiction isn't bad. But for the love of reading, skip anything new and fashionable until enough decades go by for the influnce of fashion to fade. And skip things that aren't enjoyable to actually read.)
I am in complete agreement with you that 99% of books are crap. It's the 1% that you hopefully want to get around to reading, and those are typically not the easiest reads.