The number of bug fixes to important programs has skyrocketed. Look at what Google are saying about how many Chrome security bugs they've been fixing lately. Other big software firms have been doing the same thing - AI has been finding and fixing a ton of bugs. I know of one big program where thousands and thousands of security bugs are being found and fixed.

It may not feel like this to you because a lot of the dollars right now are going into security bugs which you can't perceive. But it's definitely happening.

At the company I own I've got AI employees autonomously triaging backlogs and fixing long tail bugs. The software is definitely getting better, although by definition long tail bugs aren't ones you are likely to encounter. The subjective "feel" of how robust the software is won't change quickly.

New tech is always applied in apparently boring ways because we are imagination constrained and people harvest the low hanging fruits first. Remember claims there was worldwide demand for only about four computers? When Gates said he wanted a computer on every desk and in every home people laughed at him. What would people do with all those computers, they asked. But he was right about where the world was heading.

There are several hundred thousand mathematicians producing hundreds of thousands of new results in math each year. Why can't most people name any human contributions to mathematics from the past decade? What is the finished result that affects everyday life? Do you consider all those human mathematicians to be useless?

Most of the biggest breakthroughs in mathematics, breakthroughs that win Fields Medals like sphere packing in dimensions 8 and 24, have no applications in everyday life. Probably the only new mathematics results that people notice affecting their daily lives are the ones that enabled AI.

Nevermind mathematicians. What about the millions of programmers? Are they all hype too because people pre-2023 were griping on hn that software is buggier than ever and people are still using the same operating systems as always? Why couldn't the 30 million human programmers make something better in the past decade?

You set your bar so high that all the world's human experts in math and programming combined would fail to meet it.

The top LLMs in 2024 were Sonnet 3.5 and GPT 4o. You couldn't have expected those much weaker models to be making breakthroughs in math. The models that are making breakthroughs haven't been around very long.

We're not going to suddenly have new robots or operating systems. Those things take time. Just because there's massive change afoot, doesn't mean it's widely adopted or applied at lower-level products. ChatGPT is a product, and software, and a breakthrough. You can have a freeform conversation with your computer about anything, in human language, and ask it to do or make stuff and it will at least try, sometimes with surprising results. That wasn't possible until recently. The robots and products are coming, rest assured.

Amazon’s chatbot processed a price adjustment the other day without forcing me to call or chat with a human agent.

Constructed human life is just more complicated than the AI capitalists would want you to believe. For example, even if an AI model can design a circuit-board, does that mean it's inherently useful? You need to source the wafers, cut them, package them, advertise them, etc. Given LLMs by their nature are confined to language and language-adjacent tasks, that is a very small percentage of the overall reasoning needed to make changes in the real world. In reality, LLMs are the intended way to extract maximal surplus-value from white-collar workers. We may see an increase in innovation as a result of that, but not because AI necessarily did it, in the same way that the power loom didn't create computers because its textiles clothed the computer scientists.

I talked to an LLM at burger king the other day until it couldn't figure out that I wanted to change the drink on my previous order.

"Low code" has been a dream of the industry for longer than I've been alive. There are reasons SQL and COBOL look superficially like English even when it's inefficient to do so. There are reasons Excel is the most popular programming language. Programmers have always been trying to enable non-programmers to write software.

Experience, culture, and domain knowledge are still somewhat of a moat. That foreign youth is unlikely to be able to write a good prompt for building, let's say, the software in an FDA-regulated medical device or custom Fortune 500 ERP application. The LLMs are great at building what you ask for but it's still garbage in / garbage out.

I don't know if he's right, but Peter Zeihan thinks the breakdown in globalization will negatively affect the ability to continue to improve the chips that AI depends on[0]. Too many steps in the supply chain, too widespread, too vulnerable to deglobalization.

0: https://zeihan.com/the-ai-race-to-regression/

Maybe, maybe not. It's not unreasonable that these systems cap out at some point, or perhaps fizzle away entirely.

The businesses that create these systems are not profitable and run at a massive historical and go-forward loss.

New data centers required to operate these systems are facing increasing pushback at local levels. New construction is not guaranteed. Energy and power grid constraints exist as well.

Government regulation is way behind. What happens when (if) mass layoffs due to AI occur? How does the population react? Theoretically AI can be regulated out of significant progress, or outright existence for many purposes. At the end of the day, US and other prominent governments make the calls, not corporations.

For sure, and for that reason I mean to say that I wouldn't feel great as a manager/executive/etc either.

Those nuclear powered flying cars envisioned in the 50s were also inevitable progress of the automobile.

In the US, Business' are treated like people with free speech rights. If it would be cheaper for them in the long run to use ai and robots instead of humans, they will figure out a way to make it so.

For the moment that may be true. They are getting better and better at acquiring, retaining, and processing domain knowledge. I wonder what this will look like in a few more years.

The responsibility side is a different matter of course.

>There are two things SOTA LLMs fundamentally cannot do.

I would say there’s a third thing. They seem to be very bad at being creative. Maybe they will eventually fix that, but if you ask it to come up with a list of business names or business ideas, for example, what you’ll get is the most generic, boring answer you could think of. They seem to be terrible at extrapolating outside of their training data. To me, this is the most significant difference.

> they cannot learn new things without forgetting things

Where did you get that idea from? Basically last few years was them constantly learning new things while improving their capability on the things they already knew.