A few hundred lines of text is nothing for current LLMs.
You can dump the entire contents of The Great Gatsby into any of the frontier LLMs and it’s only around 70K tokens. This is less than 1/3 of common context window sizes. That’s even true for models I run locally on modest hardware now.
The days of chunking everything into paragraphs or pages and building complex workflows to store embeddings, search, and rerank in a big complex pipeline are going away for many common use cases. Having LLMs use simpler tools like grep based on an array of similar search terms and then evaluating what comes up is faster in many cases and doesn’t require elaborate pipelines built around specific context lengths.
When I last tried this with some Gemini models, they couldn't reliably identify specific scenes in a 50K word novel unless I trimmed down the context to a few thousands of words.
> Having LLMs use simpler tools like grep based on an array of similar search terms and then evaluating what comes up is faster in many cases
Sure, but then you're dependent on (you or the model) picking the right phrases to search for. With embeddings, you get much better search performance.
With current models it's very good.
Anthropic used a needle-in-haystack example with The Great Gatsby to demonstrate the performance of their large context windows all the way back in 2023: https://www.anthropic.com/news/100k-context-windows
Everything has become even better in the nearly 3 years since then.
> Sure, but then you're dependent on (you or the model) picking the right phrases to search for. With embeddings, you get much better search performance.
How do are those embeddings generated?
You're dependent on the embedding model to generate embeddings the way you expect.
Gemini 3 Pro fails to satisfy pretty straightforward semantic content lookup requests for PDFs longer than a hundred pages for me, for example.
Your original comment that I responded to said a "few hundred lines of text", not hundred page PDFs.