I think you'll be pleasantly surprised about what kinds of things people look favorably on.
In my early career years, a fellow employee came to work in track shorts and flip flops. He was a very, very good programmer. But he never got raises, and never got promoted, and complained to me about it. I suggested it was the way he dressed. He said the same things you wrote.
A couple decades later, I ran into him again at a conference. He ran his own quite successful company. He also was dressed sharply.
Things that make you go hmmm....