Looks like they tested that in the paper, and the code allows for it as well. https://github.com/facebookresearch/context-language-models/... Of course, this is for suggesting context management strategies rather than the actual management afaict.