Just going to sound really pedantic here, but RGB does not capture the entire colour space. In fact, it only captures about 35% of the colours the human eye can perceive.
replyhttps://www.oceanopticsbook.info/view/photometry-and-visibil...
You seem to be conflating "RGB" with one particular RGB color space: sRGB. That's a common enough conflation to make, but not appropriate when you're trying to be pedantic.
replyDoesn't matter: there's no RGB model that captures the colour space. That exactly the reason CIE exists.
replySince you seem to know, and I am curious, doesn't CIE[1] effectively use RGB to describe its space, too? Eg: the r̅(λ) g̅(λ) b̅(λ) color matching functions? Or is there something else in CIE you're referring to?
replyOkay, but that was a really useful metaphor if incomplete in a lot of ways. It made me say “oh”.
replyThis is solved through https://en.wikipedia.org/wiki/Primary_color#Imaginary_primar... that sit outside the visible spectrum.
replyLast statement is a bit sus... Muon computes matrix sign function which can be defined as setting singular values to 1, though you can also define it without SVD. Muon itself doesn't use SVD because it uses a faster method to compute matrix sign. Adam doesn't do anything related to SVD or singular values. Also not sure what you meant by "second order singular values"
replyADAM is related if your second derivative matrix happens to be diagonal.
replyOf course, it takes about 5 minutes to show that any DNN is going to have very very high magnitude off-diagonal terms by the way it's constructed, so pretending that a diagonal approximation is close enough is crazy.
Adam doesn't use the second derivatives matrix, it uses second moments of the gradient, which is the diagonal of the uncentered covariance matrix, but neither of them are directly related to SVD or singular values anyway.
replyThere is a slight connection where Adam approximates full-matrix Adagrad which computes inverse square root of the convariance matrix, which you usually do using eigendecomposition, but on the covariance matrix SVD and eigendecomposition are equivalent (can easily be converted to each other), so you could use SVD to compute the inverse square root.