upvote
This sort of result is about steady progress. It's not practical, yet. This system only allows for 1k context, not even enough for a system prompt. But the authors have contributed several orders of magnitude of speedup on the current state of the art.
reply