It’s predominantly a way to shut down thinking, which is why it’s so often misquoted. Knuth’s version is more nuanced. Theirs is not at all.
As a compromise I got really good at doing optimizations that also improved readability. Things like hoisting and better variable naming combined together can be mistaken for just code cleanup.
I also discovered that keyhole optimizations are more palatable. Which is to say, move the slow bit yo a separate function with a single responsibility. Then do the optimization in a separate commit. Then the other devs don’t have to look at your changes and if they turn out to have a bug, we can always revert the change later. They almost never do, but having the option seems to grease the wheels.
Once I got very confused when exactly this kind of 10k x speed up I demonstrated for a problem introduced to me as a business bottleneck then got ignored (I wasn’t on this team, though, but it’s still hard to understand the mentality and incentives there).
My rule of thumb is that as long as it doesn’t really complicate or convolute the code, write with performance in mind.
My expertise is in C# and sometimes this is as simple as using a hash set or dictionary instead of a plain list.
Other times it’s having a decent understanding of how EF (an ORM) actually works (the static typing benefits of them in C# makes ORMs very valuable when used appropriately) and the cost of round trips to any network host (including a DB).
To you, 2x faster means 2x the work done in the same time, but 25% faster means 1.33x the work done in the same time.
I see this a lot - in software people interpret 50% faster as a 50% reduction in time taken.
If a car is 50% faster, I hope people would universally interpret that as e.g. 20 to 30 mph. But if I asked people how long would a 1 hour journey take if it were 50% faster, I bet a lot would say 30 mins, because it kind of seems like that's what I mean.
This is not the right choice for everything, just as using Python is not the right choice for everything. The question to ask is: what are we trying to do in the first place? Then take it from there.
You did that by making sure the speed-up was business relevant. I'm mostly putting this out to force people to think about whether that also applies to their specific thing.
"Unclean" code for this is putting objects in a table, giving objects a generation (that increments whenever you create/delete one) and storing pairs of (index, generation) numbers everywhere you need a link. Link is live only if the generations matches and since only odd generations are live objects, default construction means no link. Lifted straight from 2000s video game tricks and with modern C++ it's basically all encapsulated away.
I've made that refactoring in a codebase, not because of speed but because of maintenance and memory fragmentation concerns. Sometimes, the clean solution really isn't the right one and it's really sad that compsci only ever teaches classical data structures, like we're still coding in the 1970s on computers with no caches that don't suffer from pointer chasing.
I see what you did there
I think cache friendly code also tends to be reason about friendly and debugger friendly. None of that is remotely premature optimization.
Not sure what examples you have in mind, but the very first things I think of when I hear cache friendly code are cache-blocking and cache-oblivious algorithms, which are quite the opposite of that. See also B-trees vs. binary search trees, etc.
Calling everyone who downvotes you "incels" is very much against site rules. You've certainly been here long enough to know that. Knock it off.
A "real performance bottleneck that I observed during testing" is a little bit unclear whether it occurred outside of test conditions, or whether speeding up that particular bit improves anything anyone cares about.