It goes entirely against the culture of research mathematics to hoard secrets. If you've done something interesting, you want to share it, and it's in your best interest to share it.
Of course, this culture could change under external pressure from AI tools. But, aside from cryptography and the applied fields, I'm finding it difficult to imagine any possible motivation for spending substantial effort on top-level math and then keeping it secret.
(Disclaimer: I had quite a bit of formal training in mathematics at one point, but have worked in software for more than two decades now. I wasn't on the applied side, but I recall that MATLAB evoked similar concerns back in the day.)
> It goes entirely against the culture of research mathematics to hoard secrets. If you've done something interesting, you want to share it, and it's in your best interest to share it.
You're describing an enviable human culture with a reward system suited to people. People publish papers etc. to enrich the human domain and to get recognition. AIs don't care about things - and neither, I'm asserting, will their owners. That culture will become a vestige.
Mathematics as the underpinning of science and technology that can be exploited for inter-state competition, including cryptography but not limited to that (image analysis, targeting algorithms, conflict modelling, whatever) won't see the light of day. The AIs that discover the mathematics will be paid for by the warfighting budgets of states, and by the corporations that feed on them.
If you can derive some special formula that prints money from accurate prediction of asset valuations if and only if you're the only one with the efficiency, you're incentivized to keep it quiet.
If you're a nation state that's cracked a simple method to decrypt modern "pre-quantum" encryption schemes with less compute/wall time using a development in mathematics, it is in your interest to keep this quiet so that the parties you surveil to not switch or adapt. Similarly if you've figured out a handy new form of encryption, you're incentivized to prevent leaking of details as it is harder to reverse such operations without details of the methodology.
Or you've designed some special optimizer that you then bundle in a subscription service. Your business now only profits while the algorithm is secret. e.g. The DVD Copy Control Association's CSS encryption was only a valuable thing that the company could charge for or entice clients with before DeCSS was published/became popular.
Update: see also the later slide where he problematizes corporate secrets in relation to math culture.
Everyone should just read the slides, really
People on here have also argued with me that enjoyment and fun should be decoupled from work but I'd argue that jobs that are enjoyable are important because I mean, what really counts in learning is inspiration. If no one is having fun, no one will be inspired to learn and do math in the first place. The best professors I've had are clearly having fun.
And sharing that fun on a human level is exceptionally important.
I just don't see the tradeoff (more math at the expense of people being able to have fun expressing themselves), as being worth it. The whole point of discovering knowledge is to enjoy the process and share that with the world to create hope an optimisim in it, not to product like a factory.
Unfortunately, this website tends to bring in the rare type of person who loves to believe that they can enjoy things all by themselves, and maybe they can, but the problem is that they (not necessarily you) foist their unusual view of the world on the rest of the world by developing and supporting AI and get defensive when someone points out that it actually contributes to a horribly destructive social structure - a cognitive dissonance most of them can't handle.
This is the crux of everything you've written and I strongly disagree. The purpose of knowledge is the sheer power it brings, and if there's factory scale churning out of it, I'll gladly take it.
As a closer analogy, consider this probability (I'll be stealing from the game Stellaris here): let's say a few hundred years later, when we are spacefaring, we discover a vast cache/library of knowledge of an alien civilization, we won't just pass up on exploiting it because we need to "enjoy the process of acquiring it on our own" or "derive the satisfaction of sharing the discovery with others". No, I and I daresay most others will take as much as we can without restraint in a quest to make all our lives better/more enjoyable.
And if that's ok to do, using machine intelligence to mine/grab any new knowledge or new math as Tao is suggesting here should also be acceptable and welcomed.
A dismal prospect.
When knowledge is acquired at a human scale, and under the influence of a human culture, then extraordinary things can happen. Agriculture, the transistor, Gödel's theorems.
When knowledge is "churned out" at "factory scale" then human flourishing isn't even a by-product.
Actually, this is the fundamental divide. There are a group of people (like you) who want the world to be one way, and there are a group of people (like me) who think that world is abhorrent. I don't think we should go after that power.
But, I'm glad we got to this point in the discussion because it shows this irreconcilable divide. The truth that most people will never say is that these two groups are enemies and will forever be. They fit into this world because we currently participate in the same global system, but because of this situation, it's an unstable one. And AI is exactly the system that is distilling this antagonism to its limit.