I was thinking this exact same thing. A mom-and-pop shopkeeper or diner owner knows this level of detail about all their regular customers, and did 50 years ago as well. They could predict "Steve always comes in about 3 times a week, orders eggs and toast in the mornings, and usually just gets a Coke when he visits in the afternoons" -likewise would know something might be wrong if your visits suddenly dropped in frequency.

People are concerned about the scale with this type of thing. I don't know how rational it is. It seems like people imagine themselves in a situation like Sandra Bullock in "The Net" where if an evil and very powerful entity wanted to cause you harm, boy what a detailed picture they could weave! With this, I both disagree with them (most of us are orders of magnitude less important than we'd need to be for someone to care to do that) and agree with them (it's probably bad if tracking is so good and pervasive that the few who have a good reason to be paranoid can't avoid having digital footprints of way too high fidelity to hide).

Or they're just worried the ads are too good and they know they're so weak-willed that they'll be manipulated into shopping too much.

It is the fact that the data are used to influence the behavior of customers.

There is a big step between “we observe the behavior of user segments and predict that the company will sell x units at this time and place” and “these individual users behavior is such that we can influence them to purchase what they otherwise might have not purchased if we perform the following actions.” That is behavior modification on a large scale.

If you’re interested in developing a more robust view of this topic, I highly recommend “The Age of Surveillance Capitalism” by Shoshanna Zuboff and “We Have Been Harmonized” by Kai Strittmatter. They go into detail about what these systems are designed to achieve on a population scale.