It seems, that the amount of information that "typically" an person would want to work with, can be handled with pretty much any technique by the compute that most of us have, at a performance level that is satisfying.. It's only when you start to increase the information volume, decrease the compute, or increase the performance expectations, that it is really impactful to start to find methods that really make the most of the deep structures of that information to query it. Complexity of the original information and what accuracy means for your use case also comes into play I suppose.

I've been experimenting with hybrid memory retrieval using SQLite FTS5/BM25 for lexical search and dense embeddings from Nomic Embed Text v1.5 (768D), all-MiniLM-L6-v2 (384D), and Qwen3-Embedding (0.6B), with NumPy cosine similarity, Matryoshka-style dimensional truncation (64D → 768D reranking), and optional graph-based candidate expansion. In one 40-query benchmark, combining retrieval channels improved candidate recall from 30/40 with compact vectors alone to 40/40, although Top-1 accuracy was only 33/40. Another metric I was looking at was number of final candidates found to filter. If its very high, you can get better accuracy, but at the expense of resource use, and it can also lead to worse accuracy if your tying to choose top result efficently.. it makes a needle in a needle stack.

> I found that just building a BM25 index, and giving the LLM a BM25 "search tool" and casting a "wide net" wins every single time compared to learned methods

Funny, I did exactly this for my last RAG project. Works great for a small dataset.

> found that just building a BM25 index, and giving the LLM a BM25 "search tool" and casting a "wide net" wins every single time compared to learned methods

I've been saying this for a while now. Lexical is approximately the same thing as semantic when a frontier LLM is driving the search terms.