You're right the "commercial" timeline of Python would have supported Python 2 forever because big paying clients demand it but I think that would have led to a worse outcome for the OSS community.
It's not “by far”, the weird part is that they broke backwards compatibility but also didn't fix some of the other issues and many of the fixes didn't need backwards compatibility to be broken and most of all, the way Python works would allow code written in multiple versions to interact and they should absolutely have allowed that and allow Python3 code to call Python2 modules.
> You're right the "commercial" timeline of Python would have supported Python 2 forever because big paying clients demand it but I think that would have led to a worse outcome for the OSS community.
The cost of switching was ridiculous and it still is. “venv” needing to exist is symptomatic of the bizarre ecosystem and developers being expected to maintain like 6 different versions of the same library at times and when they don't as user you are screwed.
With Rust, multiple editions can just compile together. Crates written in one edition can be called from code written in another edition.
Also, the optimism just highlighted how out of touch they were, that they thought people would actually just rewrite everything in 5 years is bizarre and suggests a fundamental lack of understanding to me. Same thing with Wayland and GPLv3 to me. This is so common in the FOSS world. People come with a new version and just expect that everyone will quickly switch because they consider their new version better, not realizing that A) even if others think it's better, they still won't “just switch” because that costs time and money and B), very often they don't even think it's better which they never bothered to even investigate before they got started. They just assumed that everyone will agree because the “I have no idea what Xfce is or does, sorry.” people are completely out of touch with the world to be honest and it's pretty clear they're often living in some very specific online bubble and don't understand the concerns of companies, users, and just everyone outside of their own little bubble.
It is the fundamental difference in the developer's origin/experience in FOSS projects vs commercial software. However, there is an even bigger paradigm difference underneath. Academia survives on one-off projects and FOSS is often recruits students in universities who contribute as hobby, without contracts. The usual POSIX software is all about exposing things with minimal abstractions. It makes developing and iterating on software easier on the developer.
People who work in commercial software are forced into decade-long support contracts. So they develop skills in managing long-term backwards compatibility. This comes with the abstractions and software designs with more thought into their future.
You see that in Windows components that the engineers thought about how the computer systems will evolve and allocated some slack to extend the software: https://faultlore.com/blah/c-isnt-a-language/#case-study-min... . This kind of hindsight isn't something one can easily learn without being forced into the backwards compatibility and getting burned before or mentored. Some of the skills also means that you need to learn where to place the "narrow waist" of the API.
To illustrate the difference I'll actually compare Win32 vs X11/Wayland. Win32 is a much better abstraction and much better "narrow waist" point than X11 was. X11 was designed tightly around 80's hardware and people had to add hacks on top of hacks to actually make things like GPUs work. X11's network transparent architecture actually made implementing direct rendering harder. While Win32 already had some ideas that made into Wayland and it designed nice abstractions for allocating windows, direct rendering buffers and compositing. Win32 had baked in concepts like a window or a button or text rendering. So when the devices with HighDPI appeared or when new security boundaries needed the higher level abstractions of Win32 made it possible for Microsoft to make smaller API changes while the whole X11 started to collapse with 90s hardware and then was useless for 2010s.
I think you are pointing out the conflict that can then arise, but that I don't think is avoidable: a company has relatively short-term aim of surviving until year-end (or quarter-end) and making money. FOSS contributors are hoping to create something worthwhile by whatever value system they adhere to (elegance, public recognition etc).