Yes. I believe that using CVSS scores as a first pass to decide which vulnerabilities to review is risky.

The most boring reason, even if you take CVSS scores at face value, is that in many cases it is possible to leverage multiple "low" severity vulnerabilities into a massive impact.

But the bigger reason is that CVSS scores are all over the place, and the people operating roulette wheel that generates them do not have any insight into any specific person's systems.