upvote
This isn’t quite accurate. Data weighting is quite important in pretraining.
reply