upvote
I use TensorX (EU hosted) and get 330TPS
reply
I pretty reliably get 200 tokens per second on deep infra.
reply