Yes, I’ve thought about this too. The strength of these models is that there is a lot more knowledge encoded in them than the average scientist has in mind at any given time. That means they can explore many more possible combinations of concepts.
replyIf we imagine a set of all human ideas that these models have access to, then the set of possible discoveries would be something like the superset of all possible combinations of those ideas. I think all LLM discoveries are bounded by that space.
Looking at the recent OpenAI math discoveries, that seems to be pretty much what happened. Existing ideas were used as building blocks, the model found a valuable combination, and the result was something new that had real value.
Yeah and I think even what might be the most useful is extracting how Codex got to a particular solution into a skill or into even more customized software so it can be applied in lots more places.
replyI know people are working on these things I just haven’t seen the right way yet. Like in manufacturing right now people are trying to encode what skilled machinists do into software and scale it up, we need to go further on that for Math/ data analysis etc.