It was so long ago that the specifics have faded, but I remember I was coached to use a variety of positive responses. "That's a great question," yes, but also things like "I'm glad you brought that up," and "I was hoping someone would ask about that!" It wasn't my cup of tea, too artificial, but the advice was contemplated.
replyThe next question (which is a great one, from what I understand) is: Why do LLMs use these phrases so much if humans rarely use them in written form? Maybe a fair portion of training data comes from lecture transcripts, where such responses are common when responding to direct questions? And/or system prompts are just instructed to be like that?
> Why do LLMs use these phrases so much if humans rarely use them in written form?
replyAs far as I understand, it's due to RLHF. The reviewers the AI companies use don't necessarily know what kind of question is a good one, so when the LLM answers "That's a good question!", they tend to rate the answer higher because they like being flattered. Proxy models that are themselves trained on RLHF inherit this pattern. Similar effects contribute to sycophancy.[1]