Its akin to judging a man by the color of his skin instead of by the content of his character.
Thanks for your comment that clarifies how genetics provides a much more granular window into an individual than any physical characteristic taken by itself can provide.
Or it could be discrimination. There are studies that show that black and white mixed races children did not score any higher than children with two black parents, while white people score higher than black people on average, which may point to some discrimination impacting IQ.