At least historically, we haven’t really been able to tell what will be transformative in terms of real world applications ahead of time.

I had to catch a stray cat recently and take it to the vet for an infection. It's healthy and spry now but ever since then it's been deathly afraid of me.

There is tons of evidence against that. Someone armed with just LLM, can have much better uderstanding of problem then "meat brain expert" who studied the subject for decades.

We seen in last decades how "experts" are clueless, and how they predictions work.

> don’t really care whether LLM’s can produce code more and better than me

So on one side there is monopoly on "deep understanding", but on other no one really cares about quality?

For the collective (humanity, mathematical/scientific community etc). Math is not done in isolation, and if somebody does so then feedback to the community does not work as well.

Mathematics, and basic science to a degree, face the issue that they create their own problems and paths through this kind of tranformation, where external feedback is secondary. It is not as if "I want to build an app/car/robot, I let AI do it". It is as if you decide to let AI decide what to build for you and how to build it, and you do nothing at all. Instead, the pursuit of understanding is the goal itself, and through the course of humanity we have learnt that this understanding can also be useful, but this is not necessarily guiding how this understanding is gained.

Most of the contexts people here have in mind are when problems are well and externally defined. Cure a disease, optimise an engine, make an app that does X, etc. This is not exactly the case in theoretical math and never was really.

I have dyscalculia so the math world is dead to me, I can't even add 2 numbers together in my head, but I've read here on HN many times over the years people describe some version of new maths frameworks tangibly changing how they view the world. For me, I went to film school - I have a natural ability to think in movies and pictures, memory is very visual for me, before film school I had a good sense of shape, colour, light. After film school and to this day some 20 plus years later, I mostly look at the world starting with the shadows. Russian speakers, whose language has separate basic words for light blue and dark blue, are slightly faster at telling those shades apart. Speakers of languages that use compass directions instead of "left/right" appear to develop a strong, constant sense of orientation.

These things can only happen through the process, so for who? For people who want to live richer lives, want to think differently, and for you, to be around people who have such.

For math, the value of proving a theorem is often not proving the theorem (which most people already believe correctly to be true or false), but the path taken there, the new math invented, and how it can be applied to other problems. The end result is therefore almost inconsequential in moving math forwards.

I have no issue with LLMs aiding human scientists, but imo we absolutely would need to understand your hypothetical advancements completely and thoroughly before surrendering any agency to them - the LLM’s risk analysis will not reliably reach the same conclusions as a democratic human society

If we imagine Super Intelligence, where NO human is capable of understanding, then how would it ever be possible for any human to identify what is actually beneficial or not?

This resolves in a paradox, common to all magical thinking. You can certainly wish that some all powerful benevolent entity will solve all of your problems for you, but it is not likely to work out well.

None of this is new. It is the same delusions as alchemy and the same thing that tales about genies warn of.

For the upper crust of our society, work is entirely optional. Presumably people therefore continue to work for the process or the reward of the doing.

Most pure mathematicians are no more interested in ‘producing commodities’ than any other academic is. That’s not what the subject is about — at all. The confusion arises because mathematics turns out to be extremely useful (no surprise; it’s quite useful to have a detailed understanding of the basic principles of reality).

Again, this seems to be an alarmingly common fallacy here on HN. As a commenter above observed, pure mathematics (and that is what we’re talking about here) is in important ways closer to the humanities than it is to other sciences.

nobody who has ever achieved notoriety in any intellectual field ever did it to "meet demand." if your goal is to be an interchangeable widget that produces value as part of a corporate machine that exists for the enrichment of your shareholders, you are certainly free to choose that path, but don't imagine that is the limit of human existence. also, don't be surprise when you are replaced by AI, because it is a superior widget.

Unfortunately many countries like the US are very under-developed, and have trained it's citizenry to approach education as job training. This is obviously an abject failure, and probably a contribution to our anemically slow and pathetic growth rate.