That's the nature of research. You try every idea that may be a good avenue and only a handful work out, if at all. That's why quantifying research credibility via publication and citation counts inherently lead to toxic work cultures. The best ideas must be given time to be discovered, not forced out and contorted to fit the requirements of a journal.
replythis is part of why I think most researchers get less productive over time... Someone gets some big result during grad school or early career, get some big job from it, and then struggle to get new results of similar quality :shrug:
replyWith ML in particular, there's also the sheer volume of people basically all looking at (essentially) the same problems... so it's kind of like monkeys with type writers spamming ideas until some work.
It's not just ML research; that's just human nature.
replyWe like to see hard-working, God-fearing people minting raw knowledge from Mount Olympus itself, whereby each shard of crystalline insight is carved meticulously by the Apprentice over the course of a productive and morally pure career.
The reality is it's some skill plus the occasional drive-by of an unknown force of nature, hitting you on the head with a shattered fragment of insight whose provenance you'll remain completely ignorant of. I'd say we just revert back to invoking the muses. It was a fine explanation.
In spirituality it is believed that ideas and inspirations aren't our own. That our mind is like an LLM that gets prompted by higher beings. In research everyone has high param count minds, trained for many years by studying. But just like LLMs by themselves are useless at creating new original work, no matter the compute you have available, so the mind can not create anything new without "inspiration"
replyWow, this makes ML sound even more like voodoo than I thought. Can you give examples of what the nature of these ideas is?
reply[dead]
reply