The tradesmen working on my house renovations aren't consuming SAAS products during their day jobs.
The white collar workforce can't rapidly switch to blue collar jobs.
So for these companies to remain viable, they need the white collar workers to still somehow end up with enough money to pay for services that ultimately the companies provide.
Maybe the turning point will be a recognition that companies can't only focus on maximising shareholder value. They also need to consider their role in maintaining and improving the societies they operate in.
There will be a period of rapid change. If we are lucky, the political class will see and adjust policy quickly. Otherwise we will see US urban areas gutted like the Rust Belt was after NAFTA / WTO. They are making the same mistakes but in a different industry.
What's uniquely un-automate-able about those jobs in their dream future?
Add in the fact that open weight models are 6-12 months behind frontier models means AI companies aren’t building a moat, they’re on a treadmill. And treadmills don’t justify the valuations OR the hype.
AI companies are in trouble.
Even this supposed profitable enterprise, the people involved are absolutely too moronic to be able to control the thing they try to invent, it will just be a matter of time before it turns around and eliminates them as well...
Some are piling on masses of debt to built capacity (eg. Oracle). Others are just reinvesting the profits from the rest of their company (eg. Google, Meta).
Anthropic’s moat is their best tool, Claude Code.
OpenAI’s moat is the brand of ChatGPT, once the fastest growing app in the history of the world.
It’s possible that open weight models keep pace, but it’s also possible that the investment to train them becomes prohibitively expensive and open weight models cease to keep pace with the large foundation model companies.
There is no theory that says the current frontier models cannot exist in models with 1/100th the compute waste ;). When we start trending in that direction, and oh wow we truly are, there will be no reason for these services. You could run them on your own hardware without serious investments.
The moat openai and anthropic have is them among others have attempted to buy all of the computer hardware for the next two years. That's intentional. They know the only existential threat to them is anyone coming up with a way to do this better than them. It's already happened and it's going to become more and more divergent.
It's not a theory. These smaller models that are coming out are huge advances for the field.
I can't comment on companies training practices. That would be proprietary stuff I guess. I think the claims that the advances being made are due to distillation alone are completely unfair. The advances alone are not just data.
Chinese megatechs stole copyrighted data AND trained their models on derivative / synthetic data that came from the US foundation models.
I’m happy Chinese foundation model trainers were able to use Huawei (homegrown) hardware to train their models (also because having Nvidia dominate that sector is terrible for competition), but if Chinese megatech companies are just deriving their open weights models from US companies, then this is just an IP theft exercise.
I haven't read the claims, so I don't know how easy it will be to work around them. This particular one seems to cover encoder-decoder networks, so it's not necessarily applicable to later LLM implementations. But I'd be amazed if Google didn't have several other relevant patents in their arsenal.
the entire US economy rides on this now so it’ll be more than few people and a lot more than few percent.
Robotics isn't even 1% of the way to replacing anything.
Consider why every neat demo is a backflip and not washing the dishes or laying bricks or something.
It's a breeding ground for Edisons and Morgans, not Teslas. It is profoundly depressing that SV is doing everything it can (knowingly or unknowingly, not sure which is worse) to get the entire planet to stop taking it seriously and to shun it.
In all seriousness, the silicon is still designed in Silicon Valley but maybe you don't hear about that as much? Broadcom, Qualcomm, Intel, Samsung, AMD, Nvidia, etc. all have a huge presence there still.
Just to emphasize my point, China is not being deprived of chip _designs_ (via export bans of ASML-made lithography equipment), but rather of the actual physical machines that rearrange the atoms.
We buy food and go for walks as entertainment. It's been refreshing but also obviously scary.
It’s the combination of AI changing the workplace, the large techs shedding double digit headcount, recruiting / hiring departments being so broken by the AI arms race hitting job applications, and the macro business environment generally being on the downward slope at the moment.
This paper proposes a solution for which the framework/base is missing.