Pi can only log what the model shows it. Many models keep their thinking traces hidden and only provide a hash or something to recover it on subsequent resumes. DeepSeek shows CoT traces, and is maybe the best model that does so, I think? Kimi stopped providing CoT traces a little while ago in their subscription service via Kimi Code, I believe. I haven't checked GLM or Qwen 3.8 Max, though I guess if you're hosting the open models yourself or using an alternative inference provider there's probably got to be some way to get at that data.

Anyway, this particular harness isn't doing anything unique, but the combination of an official agent intentionally keeping the data and making it accessible to the user and a model API that provides all the information is unusual and worth calling out. It used to be common, most APIs and models and agents showed the reasoning, or could be configured to do so. Most no longer offer it.

Asking them what they think is happening is actually not reliable, though? They don't always know how they came to a conclusion after the fact. It is probable that it's roughly similar to the path they took to get there, since it's the same weights, but it's not certain. And, if you make it standard practice to always collect that data (e.g. if you have an automated tool to ask the model to explain itself after every action to log it), it seems like you might find yourself being blocked for violating terms of service. It looks like "distilling".

In short, there are workarounds, but they're not guaranteed to work forever and they're likely to bump into terms of service.