That's more-or-less what RAG is; a tool to surface relevant data so that AI can make informed decisions. Efficient retrieval is important so that AI doesn't need to ingest the whole corpus for each query.

latency is often the reason, distilling interactions into significantly better retrieval is the second.

My case:

With a very large dataset of totally unnormalized data, semantic folding is a silver bullet.