32k context on my 3090 runs fine.