They have drivers in most of these cases; at a bare minimum the silicon was tested by the DV teams, and that generally includes running drivers.[0]
The issue is getting drivers upstreamed rather than just languishing in the vendor BSP.
And the answer for why they don't get upstreamed by the vendor is multifaceted. First off, the drivers in the vendor BSP are simply not at a quality level that would be accepted upstream. On top of that, even if they were at the quality needed, practically that coordination with upstream is a decent amount of work. Additionally, their customers don't really even care about upstream in the vast majority of cases, but instead prefer some vendor outdated fork billed to them as "stable".
[0] Apple for instance is rumored to have an internal Linux distro (or at least kernel fork) for DV of their Apple silicon chips to allow the hardware teams and macos teams to work with fewer cross department dependencies.
the biggest issue is that actually contributing to upstream is an *incredibly* difficult and painful process.