Eg: < Optimize for the slowest resources first (network, disk, memory, CPU) in that order, after compensating for the frequency of usage, because faster resources may be used many times more. For example, a memory cache miss may be as expensive as a disk fsync, if it happens many times more.>
This is just obvious common sense. Great read.
Plus not all speed-ups are made equal. Some speedups of a part of program can have no or even negative impact (as in making part of program faster slows down overall program execution).