Non-user-replaceable batteries are terrible. It turns devices into e-waste. I wish device manufacturers would understand that. I wish consumers would understand that.
I agree with you, also for things like radios alkalines are great as a battery that stays outside of the radio and is used when you need it, and doesn't leak much charge at all.
But e-waste is still a problem.
Device manufacturers want you to buy another one. They understand. Consumers, not so much.
No need to think about charging the controller - just have a pair of charged AAs nearby and switch them out when it becomes necessary.