OpenAI claims not to:
https://x.com/aidan_mclau/status/1986255202132042164
replyYes, Racoon on a unicycle? Magpie on a pedalo?
replydeleted
replyPlatypus on a penny farthing.
replyEven if not intentionally, it is probably leaking into training sets.
replyThe estimation I did 4 months ago:
reply> there are approximately 200k common nouns in English, and then we square that, we get 40 billion combinations. At one second per, that's ~1200 years, but then if we parallelize it on a supercomputer that can do 100,000 per second that would only take 3 days. Given that ChatGPT was trained on all of the Internet and every book written, I'm not sure that still seems infeasible.
How would you generate a picture of Noun + Noun in the first place in order to train the LLM with what it would look like? What's happening during that 1 estimated second?
replyits pelicans all the way down
replyThis is why everyone trains their LLM on another LLM. It's all about the pelicans.
replyBut you need to also include the number of prepositions. "A pelican on a bicycle" is not at all the same as "a pelican inside a bicycle".
replyThere are estimated to be 100 or so prepositions in English. That gets you to 4 trillion combinations.