The economics/upgrade math just does not make sense.
My best guess for Apple's actions is that despite there being a very real demand for a smaller phone, they don't think the discomfort is bad enough for people to switch to Android, so they don't even try. A small phone makes a lot of profit, but ignoring the demand also makes a lot of profit.
The Air was a real flop.