Statistics differ, but refugees granted protection range from a single-digit percentage of recent immigration into France to about ~15% or so (other countries have a somewhat larger share, including other European countries). It's true that many people tend to conflate proper refugees and economic migrants to whom a points-system might apply, but this is a general problem with how migration policy is discussed on all sides of the political spectrum, not something that's original to my comment.

Want to admit more refugees without endangering social cohesion? Then you should make sure that you're also carefully selecting your economic migrants as best you can. It's not a matter of assigning different human worth to each, but of simultaneously abiding by legal obligations towards actual refugees that are binding for the country, and also trying to do the absolute best you can for the highest amount of people who might be wanting to expatriate to it for different but nonetheless valid reasons - without unduly burdening that country and society in the process.