Loss of neurons isn't disastrous as long as it's not too many. Babies are born with several times more neurons than children have, and spend a few months letting all the useless ones die. Apparently it's better to start with a random tangle and prune it to shape (like cutting a box hedge) than to make a hedge grow in a box shape to begin with.
And IIRC brains can grow more neurons under some circumstances (certain drugs?) - they just normally don't.
https://www.sciencedirect.com/science/article/pii/S193459092...