Ideally the hold out set is randomly sampled. You need a hold out set of Inklings pretraining run.
Should we trust that the dataset of each company is big and diverse enough to be representative? I don’t hope too much…
ln(5,7) = 1,74
did you even use the right logarithm? because they use the natural logarithm..
edit: that puts it before GPT-3.5 even