Every model with open weights has some code which can be used to inference it. So we download the published weights and run against inference library they suggest, be it transformers, Nemo, etc