Nice.

What did you use to do this, something standard like llamacpp or something else like vllm or your own contraption ?