upvote
Show HN: TrnRT – An open-source LLM inference engine for AWS Trainium

(github.com)

Tokens per minute is a strange metric, I usually see tok/s. A per-second figure would be more accessible in comparing this to alternatives.
reply
how does this compare with sglang
reply