I have earlier argued that the sheer number of minds to be coordinated affects the cost of the effort, for a major part of the cost is communication and correcting the ill effects of miscommunication (system debugging). The Mythical Man-Month Frederick P. Brooks Jr.
Here is OpenAI's blog post about Navier-Stokes; [1] which suggests they used smaller connected swarms given the different prompts, then the findings were used to write further follow up prompts for a new group of agents.
> "Here λ is a parameter measuring how parallelisable the task is. They call it the ‘stepping on toes’ parameter. If λ = 1, you have a perfectly parallelisable task, with no stepping on toes and no efficiency penalty. But in reality λ is usually between 0 and 1 — allowing more people to help, but with diminishing returns. For example, if λ = 0.5 then 100 people working together get as much done as 1 person working for 100^0.5 = 10 times as long"
There are diminishing returns to more agents, but they're faster and can scale further than a single smarter agent - it's not really possible to just give 10k times more thinking budget to a single agent because it runs out of context, so for problems that require that much context you can ask the agents to specialize. Each only has a small view of the problem and they probably duplicate a lot of work and step on each others toes and make arcane bureaucracy but there is literally no other way to scale beyond a certain point
just like people
At the very least, the swarm would be able to set up a means of claiming avenues of thought, then verifying the solutions (or lack thereof) of other agents.
It's quite clear this process - whatever it may be - works, given the results we've seen in the past few weeks.