Dedicated summer clothes in trad fabrics are a ton less durable than their winter counterparts, though, for the simple reason that they’re much lighter-constructed. Individual pieces can be had plenty cheap if you bargain-hunt and shop used, but you cycle through more of them than, say, heavy-weight denim or a hefty tweed. Still, mine usually last a few years. Cycling them out seasonally means they don’t wear as fast as some synthetic-blend shirt you wear year-round, so you may not get more wears out of them, but they last a good long while in calendar time.
But man, do they breathe better than just about any of the fancy “tech” fabrics. And feel nicer. Durability, though, is an issue, and you have to get the fit closer to correct than many shoppers may be used to, because most of them won’t have much stretch (no cheating by blending in some nylon or whatever, like a “tech” fabric would)
I probably wouldn’t wear permethrin treated pants and let a cat sit on my lap, but “anywhere near the clothes” is a pretty big exaggeration of the danger.