>> that first reaches AGI (or whatever we decide to call it) might pay off

It is a myth. They are chasing science fiction. They either dream of an omniscient overseer who will take control of society and turn shareholders into the governing elite, or they envision a chatbot that will solve every problem, again turning shareholders into the power brokers. They forget that nobody will ever be allowed to "win capitalism" and take over the planet. Companies have are torn down for far far less. Every AI model is also data, ones and zeros. It therefore can be copied/shared/leaked. Intellectual Property law will not be the ladder for shareholder assent to lordship. It can and will be rewritten or outright ignored rather than allow capitalism to collapse.

> The thing that I'm less clear on is, if and when they achieve some kind of recursive improvement exponential growth thingy, isn't that going to heavily disrupt the entire economy / stock market? In that environment, would Anthropic's stocks really be a good bet, given they have to operate within the economy?

That's a long-term problem to be dealt with later, greed dictates to pursue the former as long as the music is playing and everyone is dancing. Similar to 2007-2008 when it was clear that things were breaking but the incentives were to play the game for as long as possible before it imploded.