upvote
Most models start to noticeably degrade around the 100k token neighborhood, which is still hundreds of pages of text that can be attended to more or less densely. Somehow it's just normalized to cram a small novelette into everything as an excruciatingly verbose system prompt.
reply
for comparison, this whole comment page is roughly 10k tokens.

how well would a human fare, trying to recall the details from ten of these?

reply