upvote
In my niche Redactle puzzle solving benchmark [1] I noticed Gemini 3.8 flash is slightly faster than 3.7 flash. They both smoke every model I've tested. I have not yet run 3.5 flash. Gemini models are great at this task because they seem to have exact Wikipedia text baked into the weights. When I rewrite the wiki text a bit it's not able to one-shot the game so much.

[1]: https://redactle.net/llm-leaderboard

reply
It depends on how you're querying Gemini models. OpenRouter is the fastest by far. I'm guessing they bought the dedicated pipe from Google. Gemini via VertexAI and consumer API has pretty bad latency.
reply
Yea I am testing through OpenRouter - have you noticed 3.7 flash being significantly faster?
reply
I guess it might be relative, but switching from VertexAI endpoint to OpenRouter was like 2-3x faster for us.
reply