Learning the basics of how LLMs work is legitimately doable in a weekend if you have ever passed a linear algebra course.
replyIf you are actually good at linear algebra, it's doable in an afternoon.
They are fantastically simple in structure, all the intelligence is in the massive piles of data in them, not their structure. Then there is of course a lot of room in optimization once the basics are done.
If interested, maybe start here: https://www.deeplearning.ai/courses/how-transformer-llms-wor...
A bit more than that. You need to understand Hessians to compute the loss function efficiently
replyhahaha I wish I was! I just do these during my free time. That being said, I am currently open to new opportunities :)
reply