This is the whole point of Karpathy's nanochat which OP refers to, to train a GPT-2 level LLM for under $100, renting an 8xH100 VM.