It's basically a variation of HyDE (Hypothetical Document Embeddings), and the rationale is that the embedding of the query is not necessarily close to the embedding of the answer. If you generate a hallucinated answer, it can line up with the actual document better (in the embedding space, via BM25, or hybrid).

But honestly, it only works for common knowledge that's already in the LLM. If the target document contains very niche or private information, then the hallucinated answer's embedding can be even farther away than the query's.

Yes absolutely that's another good trick.

Even better is to search the corpus first with like naive BM25 / embedding search, aggregate over top N to get most representative categories, then have the LLM categorize in that set.

Agreed that you almost certainly can just embed the original with most modern embedding models.

It might be a little worse, but it will definitely be way cheaper.

Yeah I had the same question. What's the point of the intermediate step?