As for tracking error across multiple color axes, I would naively assume it's a 3-vector (maybe in a linearized perceptual color space, or maybe RGB) and the error term the encoder minimizes is just the vector magnitude?
Very cool. I think it's best understood as a coding exercise for the embedded CPU vs a 'real' video format, but no less impressive IMO