upvote
> difference between an instruct based re-ranker and laya/jev I just don't see it

Main difference is that laya/jev/et-al give you a zero-shot classifier that requires no training. You can prompt engineer your way to a quick fairly reliable cheap enough decision engine that you can use to iterate quickly (by prompt engineering).

Right now a lot of people are doing this with LLMs and it's too slow and expensive.

Imo the right iterative approach to productionizing these systems is something like:

    1. Build it with an LLM. Iterate on the prompt
    2. Start building a real-world dataset
    3. When the prompt works, turn it into a clear rubric for Jev or similar
    4. Keep iterating until desired accuracy achieved
    5. Use the real-world evals you've built to train a custom classifier fine-tuned to your needs
You now have a system that has produced useful results in production from the very beginning and by the end it's a reliable super cheap classifier that can make thousands of decisions per second.
reply
I don’t think that’s it. I sincerely doubt most developers are doing side by side comparisons of calibration quality.

OpenAI has a section on their embeddings model api page for zero shot classification. Of course you can choose an open weights embedding too if you’d like.

https://developers.openai.com/cookbook/examples/zero-shot_cl...

I think Jev wins on marketing and convenience. Most SWEs don’t want to talk about embeddings, cosine similarity, or precision/recall tradeoffs. They want something which plausibly works and is easy to use.

reply
Jev's value becomes more apparent when the task is a moving target. eg an auto-mode classifier.
reply
Calibrated probability across multi task with zero shot I guess. A reranker is single task and tuning it make it even more narrow. And I guess some piping to make multiclass efficient since you cannot mask logprob for independent questions in the same output space without throwing calibration away.
reply