Yes, but the point being for the LLM to be useful to us it has to do something relevant for us and we have to define what is relevant. Even if you view them as fully human level or beyond (in terms of agency) there's still some purpose that we have to help provide them with.

To put it differently, if you have some idealized model in front of you that can do anything a team of humans can do, what do you say to it? It's still just a model that takes an input and provides an output.