update: coreweave/fp8 is at 191 tps, launched this morning, but really bad cache hit rate (~60%), coreweave is good for privacy but let's hope they improve cache
replyEh, it's also one of the providers with "blind" vision models https://mmoustafa.com/blog/so-you-want-to-use-openrouter/
replySadly, there is no way to tell if this is running with real weights or being heavily quantized.
replyjust checked, yes they don't seem to provide that information, most other providers are advertising fp8 or fp4 which is okay, but "together" doesn't, so they are likely using fp4
reply