The guy owns half the company, so a significant part of the money I'm paying is involved. Yes, it is quite ethical to decide based on matters like that. It's not an employee or minor shareholder.

Not doing business with a company (for any reason btw) is not 'punishment'. Nobody is taking away anything from the company or any people involved with that company.

That's how markets work. People have the right to choose to do business, or not, based on whatever criteria they value.