Well the LLMs are much bigger than diffusion models so I think that's the bitter lesson. You could scale compute for diffusion models though.
replyWell no, the bitter lesson isn’t the scaling laws themselves. It’s that approaches which can take advantage of scaling laws will ultimately beat ones that can’t.
replyScale with compute*
replynot explicit to reference scaling laws of training
You learn something new every day! thanks
reply