I think the recurrence is a consequence of using a residual connection, seems like that makes the representation stay consistent across layers