Perhaps they were very smart, but also they were all autistic[1], so they had poor social skills and had a hard time coordinating large groups against encroaching H. Sapiens Sapiens tribes.

[1] https://communities.springernature.com/posts/neanderthal-dna...