Specifically when the document number reaches around 10k+, a phenomenon called "Semantic Collapse" occurs.
replyhttps://dho.stanford.edu/wp-content/uploads/Legal_RAG_Halluc...
So you're telling me rampancy ( https://www.halopedia.org/Rampancy ) is real.
reply> Specifically when the document number reaches around 10k+
replyWhere are you getting this? just read the paper and not seeing it -- interested to learn more
The RAG GP used suffered from semantic collapse.
replyGemini with Google search is RAG using all public data, and it isn't broken.
replyIt's not tool use with natural language search queries? That's what I'd expect.
replyIt's RAG via tool use, where the storage and retreival method is an implementation detail.
replyI'm not a huge fan of the term RAG though because if you squint almost all tool use could be considered RAG.
But if you stick with RAG being a form of "knowledge search" then I think Google search easily fits.
It is tool use with natural language search queries but going down a layer they are searched on a vector DB, very similar to RAG. Essentially Google RankBrain is the very far ancestor to RAG before compute and scaling.
replyCant you make thresholds higher?
replyHmm... I guess not, you might want all that data.
Super interesting topic. Learning a lot.