This is an analogy, obviously. Ikea has been innovative, and it does provide a useful service for people; if you just moved into a new place and need to furnish it as quickly and cheaply as possible, then off to Ikea you go. But it's still shitty furniture.
My furniture doesn't look great, sometimes. My joinery is not perfect. I don't have all the tools I need to do this properly. But the design goals for each are what we need to live our lives. My wife has a stupidly high bed in her office, piled mattresses so she can spread them out if we have many visitors. I made her a bedside table that matches that height. It's a complete one-off; I won't make another that size, and we probably won't need it if we move house.
My point is that we already have this split in other areas of our lives; the Vimes Theory of boots (rich people buy boots that last generations, poor people buy boots every year). Ikea furniture. Buying a mass-produced crockery from a big store, or buying hand-made crockery from a local potter. We're just adding information and code to this split.
AI lacks the ability to identify greatness because it's trained on the output of the average person who also lacks this ability.
It's going to create a new elite class of people who have good taste and the masses who have bad taste. Many current elites will end up with the masses. They may retain their wealth on paper, but it will be a cheap, low-quality existence but they will be convinced it's luxury.
I think eventually, everyone will get what they want, but not everyone will get what they need.
It’s like saying robbing banks for a living isn’t sustainable and working at a bank is. That’s not exactly a stretch.
The tech world is obsessed with moving fast and breaking things, and you can't just do the same thing forever and expect it to always work.
It turns out that while these are all truisms, nobody wants to fix them. Developed countries are okay passing pigovian taxes, to a limited extent, to help fix these problems. Developing countries are even less interested in fixing these problems. It turns out that austerity is incredibly unpopular. Everyone wants to tell other people not to do the things they don't like but nobody wants to listen to what other people tell them not to do.
Just a reminder that Europe colonized Asia, Africa, and the Americas in the search for spices. Later on the interest changed to tea. Literally the only thing that Europe wanted was better tasting food and drink (initially at least.) By the time the potato had become widespread, they could have had enough calories to feed the continent, and yet the desire for flavor is what lead to untold misery for hundreds of years for millions of people.
We need to be realistic about what works and what doesn't. Austerity never wins.