Yes, TBH one of the reasons i want to try and finetune my own is to teach it stuff that now i have to explain before anything can be done :-P.
replyUnfortunately i only have a 24GB GPU - and an AMD one at that - so there isn't much i can do on that front. Supposedly a 24GB GPU is enough for finetuning a 24B model with 4bit QLoRA, though when i tried it with some finetuning app (in an official docker container) it barfed at Mistral's weird template or something and i lost interest after that.
Try Runpod or similar services: you can fine-tune stuff for the price of a latte. Stanford's NLP course recommends them: https://cs336.stanford.edu/
replyI've heard about this but i prefer to avoid anything cloud based (not just for AI but in general). I try to avoid relying on stuff i have little to no control over.
reply