This absolutely wouldn't allow us to predict what species will emerge and that something like humans would be the dominant species.
Similarly, we understand the mechanics of how LLMs work, but that does not make it predictible in any way.
We have the same thing with LLMs. We don't know how they work. We don't understand them. We have only empirical, statistical indications, nothing is proven. We know how the small parts behave, we really understand those, and also the different algorithms. But they whys? No. We're getting better, but still far away.
Basically every science field is based on this.
As far as not knowing how it works, that becomes a bit of a philosophical problem. We know how LLMs work and their algorithms. I think we can always play up "we don't know" is disingenuous ways. How many people who made Apollo calculations could also write long proofs and theorems of the fundamental maths they were using from first principles? Not a lot. We're always standing on someone else's shoulders. We'll never truly know everything or how exactly our tools work on an individual level. Maybe AI changes things because an innovation may not have a direct line to an Isaac Newton at its base, but I think that's a bit of a vague and poetic difference, and not a real one. Its still human achievement but via machinery.
I dunno, its a bit scary, but I do feel like its just continuing the same line of human ingenuity. Not knowing exactly how something came about and non-determinism scares some people, and perhaps rightly so, but pretty much all major technological advances are scary on some level because they so strongly challenge the status quo and our ways of thinking and or ideas of what is good and right and proper. At a certain point, its hard not to see the more dramatic AI outrage as just hysteria and moral panic. I find it interesting how easily we can spot moral panics in the past in history, but when its our turn, it can be very hard to see when we're in a moral panic. To many living during those times, the moral panic sounds entirely reasonable, but its still a moral panic.