upvote
The author does define these terms implicitly at the beginning of §1, where they define standard MHA in this notation. From the formula, (q_t) is the query generated from the hidden state at the (t)-th position (e.g. the token at position (t)), (k_i) is the key for the token at the (i)-th position, (o_t) is the (vector) attention output for the (t)-th position, etc. (S) is then defined later as the sum of the outer products of (k_i) and (v_i) over all positions up to (t). However, I do agree with you that it would not hurt to make this more explicit.
reply