They did it for 2 and 3, however it looks like they didn't for 4 and 5.

GPT-2: https://slate.com/technology/2019/02/openai-gpt2-text-genera...

GPT-3: https://www.itpro.com/technology/artificial-intelligence-ai/...