I would suspect some of non-optimized scenarios are eyeball network operator decisions on their networks that DNS providers and others do not have much control over. Like, Cloudflare resolves an IP that is closest to them, which is likely also the closest to the end user (and the eyeball ISP), but the eyeball ISP BGP path to that resolved IP takes a roundabout path because of their own BGP policy because $reasons.
BGP's default route selection is to use the choice with the shortest AS Path.
If your ISP and your CDN peer in some locations, but not all, you can easily run into longer latency.
Ex: customer in Seattle, but ISP and CDN peer in Portland. CDN has a PoP in Seattle but not peered with the ISP.
BGP (without a lot of tuning) will prefer to send traffic through Portland, rather than through transit in Seattle, because the AS path through Portland is ISP -> CDN and the AS path in Seattle is ISP -> Transit ISP -> CDN
Of course, CDNs try to get peering in all common locations to address this, but that's not always possible, and not always because the ISP is unreasonably uncooperative. Sometimes the best path to resolution is by targetting the ISP dns server, but it doesn't catch all the customers.