> Why do LLMs use these phrases so much if humans rarely use them in written form?

As far as I understand, it's due to RLHF. The reviewers the AI companies use don't necessarily know what kind of question is a good one, so when the LLM answers "That's a good question!", they tend to rate the answer higher because they like being flattered. Proxy models that are themselves trained on RLHF inherit this pattern. Similar effects contribute to sycophancy.[1]

[1] https://arxiv.org/abs/2310.13548