> Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established. And I believe that international coordination on future AI development needs to become a top priority for governments around the world.
I am sure it will lead to more material wealth, but I can’t help but feel pessimistic about the political economy of making millions of people redundant. What leverage will knowledge workers have in the new world?
We can look at the past and wish we were more clear-eyed about what was happening, just as we can look to the present and ask that we have more empathy for each other going forward. But I don't think it helps anybody to blame the mathematics community for not organizing until they were directly impacted. This is a tidal wave that has caught everyone off guard in <3 years.
Yes, mathematics has been perhaps the purest human intellectual pursuit. Sure, many theorems turn out to have important applications in science and engineering, but the mathematical community has mostly escaped corporate interests. And for the reasons you mentioned about not needing any resources except your brain, it has been a uniquely human activity which showed us talent can come from anywhere, with stories like Ramanujan and Galois.
I hope that pure mathematics research can retain a strongly human component forever. It would sadden me immensely for human understanding of our mathematical world to wither and die, and for us to become ignorant consumers of wonders beyond our understanding just because our robots can do it better than we can. As far as applied research goes, I hope we will always be able to understand what we want to, but I have less qualms about becoming more scalable and efficient.
>for us to become ignorant consumers of wonders beyond our understanding just because our robots can do it better than we can
all this fantasy books with magic artifacts should have mentally prepared us. Time to study the prompts Potter was giving to his magic wand.
After all, one of the main work the top AI companies are doing rigth now is developing AI to further develop AI. After several layers of AI developing AI we probably wouldn't be able to understand much there.
But you're talking about business, where competition has always been expected. OP is talking about mathematical research, which has stood on hundreds of years of cultural tradition driven by human individuals sharing ideas, collaborating, building on each others' work, all for the benefit of humanity.
But what does the human researcher do in this future?
If they are not needed to understand the result, then what's their role? Asking the right questions? But how will they know what questions to ask if they don't have a deep understanding of the domain earned by sweating the details themselves?
And how long will they be needed to ask the right questions, how long until AI can do that too?
> Software has always progressed this way
These platforms took on the order of a decade to mature, during which people had plenty of time to learn what's next. With LLMs we went from one-shotting functions in 2025 to entire projects just over a year later.
What do you think you will be working on in a decade?
> Asking the right questions? But how will they know what questions to ask if they don't have a deep understanding of the domain earned by sweating the details themselves?
The right questions are very simple to ask.
How do I get food. How to cure aging. How to turn lead into gold. How to fly high. What is the ultimate theory of physics. Are there any odd perfect numbers. Is there a soul.
We have reached complicated questions requiring deep knowledge because we tried to solve the simpler ones and reached obstacles. For example to solve alchemy we had to develop nuclear physics (and in the process we got chemistry). If one has a genie able to solve questions, you won't have to think about the complicated ones because the genie will.
I think it will be like how a lot of people know how to code in python but have zero understanding of assembly or how a cpu works. As a researcher you'll accept there's this low level stuff that if you want you can dig into but isn't worth your time, like looking at the generated ASM isn't.
You're exaggerating the LLM progress a bit but yeah progress has been crazy. Yet not much has changed right? We mostly have the same jobs, just code wayyyy more than before because things that wouldn't be worth it now are worth it.
10 years from now I have no fucking idea. But I'm sure in the meantime those who leverage AI will do better than those who yell at a cloud.
True, surprisingly not much has changed in software despite the incredible progress. I think it's partly because much software is an "open loop" system where what you build depends on running the software on client computers, devs talking to users, etc. There are still things that only human engineers can do. I worry it is less so for poor mathematicians, where math is "closed loop", and the exchange of capital into research progress is more liquid.
I can't help but be pessimistic about AI moreso than any other technology. Why? I used to be excited to learn the next big thing because I knew it would unlock much more things to do and get paid for, and there would always be more for me. With AI, the new things come nearly too fast, are not very deep or satisfying, and it's hard to see that there will always be a place for me. And so while I leverage AI in my day-to-day, I will yell at the cloud, too.
Texas is a weird case because they're the fastest growing state in the country and they run their own independent power grid --- badly (see: the winter outage a couple years back). And even then: data centers are going to get built in Texas. Gov. Abbott is doing a performative audit thing, but they're ultimately going to clear projects.
It’s because it’s from the French (if you say the plural with a French accent, it all makes sense). There are a handful of other cases in English, though none spring to mind this morning.
I feel like you’re half right? Daughters in law and passers by don’t feel Norman French. Daughter is definitely from middle German, yeah? And I am wondering about the professors example because the whole point is the plural only happens on the first word.
That said, I so want to be a part of the last group.
Tao’s key point is that the way people are using AI today does disrupt that. Because people and companies are valuing the results over understanding. So we’re getting slop results rushed out that are automatically verified.
Moreover in the past, discussion and idea sharing would happen naturally to overcome the friction of the process. But now when OpenAI is stuck on a particular part of NS for example, they can just throw more capital & tokens at the problem.
I won’t speak for Tao. But it is not “people” who have valued the result over understanding. “People” are a bit impressed that the machines are as good or better than the priests who have been praying at the inscrutable altar of pure mathematics. But it is mathematicians—I am speaking as a mathematician—who have adopted a culture of valuing results over exposition and transparency. The field has been rife with extremely opaque papers for many years and the character of research participation has been one of exclusion and competition over proof priority at the expense of understanding and transparency. It is unbelievably ironic to listen to mathematicians complain about “AI slop” when they have built careers upon human “slop” if slop means papers crammed with technical density that prevents all but specialists from reading the work.
I think it's OK that there are some materials designed for a specialist audience, and some materials designed for a wider audience. Technical density serves a real purpose in the former (I'd like to just say "stack" without including an explanation 10 times longer than the rest of my paper about what a stack is and why we are talking about them), and some people expend a lot of effort on the latter (think lectures, lecture notes, textbooks, seminars, blogs, etc--totally appropriate to dig into the motivations here).
It's just the fact that it's a really vertical field, not some cultural failing, that results in pretty opaque stuff sometimes.
> Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established. And I believe that international coordination on future AI development needs to become a top priority for governments around the world.
reply