Haven't they already proven to be extremely useful? In some areas they are definitely here to stay, coding/software and search (retrieve and summarize information). There's a bunch of places where they are surely shoehorned in, overhyped, and don't belong, but there's also equally many places where they might still be transformative but aren't used yet.
But overall I think the technology is well proven.
Besides, the marketplace is still in its infancy for LLMs, with a lot of unanswered questions. A lot of those questions surround the commercial viability of frontier models on bespoke hyperscaler data centers with limited usage outside of LLMs specifically should those economics be non-viable. Since that's where the memory is being tied up into, that means it's a critical question to answer in order to determine long-term investment needs into further memory fabrication.
Most certainly not. The accuracy issues mean that they can't really be used effectively for coding or search, the two things you mentioned.
So with compute stagnating and memory constrained, my money is on vendors taking this as an opportunity to gradually shift away from a yearly release cadence and slow down to a biennial cycle that alternates between budget and flagship launches every other year.
For example, Apple might make fewer iPhone 18 and let it sell out frequently. They’ll use their RAM supplies mostly for the Pro phones.
I don’t think Apple will stop releasing new iPhone Pros every year. The business is too big.
Apple has a second option that may not be open to most other vendors - as they've just demonstrated with the MacBook Neo, they could cut the RAM in half on the budget models. One good cycle of optimising the hell out of their (almost entirely native) software stack, and iOS would once again sing on a 4GB SKU.
I don't necessarily expect them to cut memory on the existing models, but I could well see a 4GB iPhone e/mini showing up to shore up the bottom of the lineup, as the pro models get price increases.
There's a lot of 'ifs' there to be sure, but they'd be fools not to at least discuss the possibility internally and understand their options.