With that interpretation, the issue becomes slightly different: is it more important that the collective understanding of the mathematical community should be as advanced as possible or that there should be answers to as many problems as possible? Or are those two aims valuable in different ways, so that there is no point in declaring one of them more important? Or are they so inextricably linked that it makes no sense to argue that one is more important than the other? And when we say “important”, for whom are we saying it is important: for mathematicians, or for society as a whole?
This is easy to me. Truth should be the North Star. If there is a fundamental truth that can be found via mathematics, then the shortest route to that truth should be preferred. While LLMs are definitely capable of solving problems in search of truth, I agree with Tao that instant "true/false" results threaten to short-circuit the traditional avenues we have used to escape local minima in the search for truth. Their products may be the junk food that provides immediate satiation in exchange for long-term health. Perhaps it's wrong, though.
The letter is not incompatible with “pro AI in math” unless one thinks arguing against extreme behavior such as corporations using millions of dollars to scoop results makes one “anti AI”, which is not a reasonable stance in my opinion.
AI offers the potential of enhancing and accelerating genuine mathematical study and understanding. Mathematics as a profession will need to adapt to these changes in several ways. However, whether these changes ultimately benefit the field or have a destructive effect will in large part be determined by the decisions of the humans in control of this new technology.
This isn’t true, as there are plenty of things proven beyond reach.
And plenty of things, eventually solvable, can create major problems that could both be avoided and the problem solved by taking a much better path.
Having technology and the ability to safely and sanely use the technology needs to progress together at a similar rate. The failure to do this is even a reasonable and common solution to the Great Filter. Jared Diamonds book “Collapse” has ample examples of cultures that wiped themselves completely out via not having this balance, so it’s not simply a theory.
This pattern of argument keeps repeating in every place.
Pro tech people: technology removes bottlenecks. Sometimes we use those bottlenecks as a side effect to build muscle and so on. But removing bottlenecks gives us much higher degrees of freedom. It is up to us to coordinate and make use of the technology.
Anti tech people: bottlenecks are fundamentally useful. They should remain and technology shouldn't remove those so easily. Humans cannot coordinate as well when the bottlenecks are removed, so lets not remove them so quickly.
Its not a conspiracy. Its 25 fields medalists who think exactly like how I put it. Ideally, they can just take whatever the technology gives as soon as possible and use it. They don't want it because they don't think the community can rearrange and coordinate such that they can make use of the new found degrees of freedom.
That's literally all there is to it - they don't believe in the rearrangement.
Counterpoint: I'm not anti tech at all. In fact, I sell an AI harness for legal.
I think you've set up a false dichotomy. I'd propose to you the middle ground that a lot of us are concerned that VC-backed AI slop is "solving" problems in indigestible ways that hollow out the core. This applies in OSS as well as mathematics.
Please help me understand this and I'm asking this in good faith.
Why can't OpenAI publish whatever it wants. And the math community can use it or not use it. Fundamentally OpenAI's solutions are high signal - they are incentivised to not deliberately mislead people. Let the individuals in math community choose to read it or understand it? If OpenAI wants to publish something, let them do it in the current channels using peer review using whatever time is required.
What's wrong with this? The math community thinks this will destroy previously unwritten ways of prestige allocation and remove incentives that used to exist. I say that the community can rearrange and allocate prestige and time in different ways to maximally use the technology.
I think it is easy to just roll of a local maximum, however, you may get stuck in a local minimum.
I kid, of course, but I do wonder where the use of local "maximum" comes from, what is maximum there? Why do you not see this as a landscape of hills and valleys where marbles with certain energies may indeed get stuck in deep enough holes... Of course, I just assume and picture gravity pointing down in that landscape, but hey. I'm human, I feel it is expected of me.
I thought one of the implicit points of the open letter was that unsolved problems are not something that falls out of the sky, they are a curated resource that people have spent time on and shared for the benefit of like-minded peers and humanity as a whole. And the AI companies treat them like they treat absolutely everything else: natural resources, literature, art, code etc. as something to be chucked into the ravening maw and pooped out the back as profit. They don't care if mathematics advances, they don't care if they strip-mine the available problems and damage the field. In fact, as with programming I think they see that as in their long-term interest - soon there will be no intelligence or creativity but the one that Sam Altman bills you for.
Yes, and here towards the bottom the author gets to the reason he didn’t sign with the other medalists:
I felt that there was nothing to be gained from criticizing AI companies for generating too many solutions too quickly.
Under the circumstances, I think the best we can do is recognise the changes that are coming and try to work out the least unsatisfactory way of dealing with them.
Basically let’s make it a short-term problem and deal with it. Groups of people can deal with short term emergencies. Don’t turn it into a structural issue.
And in my view what’s the alternative in the letter exactly? The tools exist. Is there going to be drama every time somebody decides to use them?
The tools exist, so let’s try to figure out has humans the best way to use them to promote humanity, and let’s advocate against ways of using them which are a detriment to humanity, which is exactly the charge the letter makes. There are cultural norms around the use of every technology.
One way that might happen is that AI disrupts society so much, or even kills vast numbers of us, that the preservation of something like the current mathematical tradition ceases to be of any concern: all that will matter is the survival of the human race. But that again is a topic for a different blog post (which in fact I am in the middle of writing).
This is pretty disappointing I have to say. The letter actually has a “pro AI” stance (see quote below), and Gowers’ apparent reason for not signing it is so subtle (not wanting to offend people whose goal in mathematics is not understanding) that I can’t help but view this as being more about optics than its actual content; i.e. he seems to be cleverer than I am in recognizing most people will not see the letter as a push to use AI in ways which are healthier for human flourishing but as advocating for one of two sides in a highly artificial binary (“pro AI” or “anti AI”).
AI offers the potential of enhancing and accelerating genuine mathematical study and understanding. Mathematics as a profession will need to adapt to these changes in several ways. However, whether these changes ultimately benefit the field or have a destructive effect will in large part be determined by the decisions of the humans in control of this new technology.
Solving a theorem is like climbing a new mountain. The mountain is already there, and there is a list of the hardest known mountains to climb. The point of climbing them and not just dropping with a plane on top of the peak is to help develop human climbing skills and expand our knowledge and abilities. Also, already conquered mountains are climbed all the time to test new strategies.
Now, if an LLM proves a theorem, it's like discovering a new mountain and knowing what its peak looks like. Does that mean the problem is finished? No, we still need climbers to actually do the work and advance the field with human understanding.
This is really a microcosm of one particular problem that AI presents to the world: what do people do when their labour is not required any longer?
Here the worry is the social structures of mathematics are eroded such that fewer humans become able to to the work and less well, similarly to how juniors are being recruited less in software engineering, breaking the ladder and leading to fewer seniors in the years to come.
The answer to this really depends strongly on what AI can actually accomplish, but I’ll assume the maximal case and say that AI can do everything economically necessary, and further even those things just desired, such that human labour isn’t required to anything that anyone wants in a practical sense.
Here, we don’t need a human understanding of mathematics to give people a perfect standard of living. We also don’t need humans involved with anything else.
Everything therefore becomes a hobby or a game. People do things because they enjoy them for their own sake, or because they are endeavours used as vehicles to socialise and enjoy others’ company, or because a shared social belief exists and is cultivated such that doing such and such a thing confers social status.
And I think that’s more or less it. I predict we may see some fairly strange sorts of things, such as games where the team structure looks like the descendant of a company org and they compete in an artificial economy. Likewise we may see gamified versions of universities and academia. All of these would be “tamed” such that the rougher parts of the experiences were sanded off.
Sort of like how we evolved in an ancestral environment, and we have certain drives and expectations driven by that environment even though they no longer matter for survival. Our social structures may be derived similarly from those of today, even after they have ceased to serve a real purpose, but changed and repurposed to give meaning and community.
This is easy to me. Truth should be the North Star. If there is a fundamental truth that can be found via mathematics, then the shortest route to that truth should be preferred. While LLMs are definitely capable of solving problems in search of truth, I agree with Tao that instant "true/false" results threaten to short-circuit the traditional avenues we have used to escape local minima in the search for truth. Their products may be the junk food that provides immediate satiation in exchange for long-term health. Perhaps it's wrong, though.
The whole point of the letter was that if we adopt this gradient search strategy the humanity might get stuck in a local maximum forever.
Thing is, you can say that about any technology (not being here necessarily pro AI in math, but I think we need better arguments).
I mean sure? It's clear PLT and other applied CS fields are now as good as dead. You're never seeing new React or Python emerging ever again.
And unfortunately mathematics is much more fundamental to human endeavor than this.
I don't think that's true. Domain experts with the desire to create and experiment aren't going away any time soon.
Sure we are free to tinker, like the guys doing model railways. With about the same impact.
The letter is not incompatible with “pro AI in math” unless one thinks arguing against extreme behavior such as corporations using millions of dollars to scoop results makes one “anti AI”, which is not a reasonable stance in my opinion.
Then we develop ways of breaking out of local maxima.
Wherever we can recognise a problem we can solve it.
Nobody is going to recognize anything if the institutions of mathematics are torn down.
Tao's point is that we have a method already.
This isn’t true, as there are plenty of things proven beyond reach.
And plenty of things, eventually solvable, can create major problems that could both be avoided and the problem solved by taking a much better path.
Having technology and the ability to safely and sanely use the technology needs to progress together at a similar rate. The failure to do this is even a reasonable and common solution to the Great Filter. Jared Diamonds book “Collapse” has ample examples of cultures that wiped themselves completely out via not having this balance, so it’s not simply a theory.
This pattern of argument keeps repeating in every place.
Pro tech people: technology removes bottlenecks. Sometimes we use those bottlenecks as a side effect to build muscle and so on. But removing bottlenecks gives us much higher degrees of freedom. It is up to us to coordinate and make use of the technology.
Anti tech people: bottlenecks are fundamentally useful. They should remain and technology shouldn't remove those so easily. Humans cannot coordinate as well when the bottlenecks are removed, so lets not remove them so quickly.
I can only suggest reading the letter with open mind and literally, without uncharitable bias towards the authors or conspiracy mindset.
Its not a conspiracy. Its 25 fields medalists who think exactly like how I put it. Ideally, they can just take whatever the technology gives as soon as possible and use it. They don't want it because they don't think the community can rearrange and coordinate such that they can make use of the new found degrees of freedom.
That's literally all there is to it - they don't believe in the rearrangement.
I can only envy the insight you have into thought processes of 25 fields medalists. Wish I could have that.
FWIW it didn't sound accusatory to me, it made me think the anti tech people might have a point.
Counterpoint: I'm not anti tech at all. In fact, I sell an AI harness for legal.
I think you've set up a false dichotomy. I'd propose to you the middle ground that a lot of us are concerned that VC-backed AI slop is "solving" problems in indigestible ways that hollow out the core. This applies in OSS as well as mathematics.
Please help me understand this and I'm asking this in good faith.
Why can't OpenAI publish whatever it wants. And the math community can use it or not use it. Fundamentally OpenAI's solutions are high signal - they are incentivised to not deliberately mislead people. Let the individuals in math community choose to read it or understand it? If OpenAI wants to publish something, let them do it in the current channels using peer review using whatever time is required.
What's wrong with this? The math community thinks this will destroy previously unwritten ways of prestige allocation and remove incentives that used to exist. I say that the community can rearrange and allocate prestige and time in different ways to maximally use the technology.
“Forever” is a long time. That’s quite the claim.
I think it is easy to just roll of a local maximum, however, you may get stuck in a local minimum.
I kid, of course, but I do wonder where the use of local "maximum" comes from, what is maximum there? Why do you not see this as a landscape of hills and valleys where marbles with certain energies may indeed get stuck in deep enough holes... Of course, I just assume and picture gravity pointing down in that landscape, but hey. I'm human, I feel it is expected of me.
Can't we get stuck in local maxima because of the lack of LLMs?
The letter is not advocating not to use LLMs. It’s advocating to use them in ways which don’t erode human understanding.
Knowledge wants to be free.
Tokens want to be paid for
Siri, can you call the bicameral order, the captain has published again..
I thought one of the implicit points of the open letter was that unsolved problems are not something that falls out of the sky, they are a curated resource that people have spent time on and shared for the benefit of like-minded peers and humanity as a whole. And the AI companies treat them like they treat absolutely everything else: natural resources, literature, art, code etc. as something to be chucked into the ravening maw and pooped out the back as profit. They don't care if mathematics advances, they don't care if they strip-mine the available problems and damage the field. In fact, as with programming I think they see that as in their long-term interest - soon there will be no intelligence or creativity but the one that Sam Altman bills you for.
Yes, and here towards the bottom the author gets to the reason he didn’t sign with the other medalists:
Basically let’s make it a short-term problem and deal with it. Groups of people can deal with short term emergencies. Don’t turn it into a structural issue.
And in my view what’s the alternative in the letter exactly? The tools exist. Is there going to be drama every time somebody decides to use them?
The tools exist, so let’s try to figure out has humans the best way to use them to promote humanity, and let’s advocate against ways of using them which are a detriment to humanity, which is exactly the charge the letter makes. There are cultural norms around the use of every technology.
This is pretty disappointing I have to say. The letter actually has a “pro AI” stance (see quote below), and Gowers’ apparent reason for not signing it is so subtle (not wanting to offend people whose goal in mathematics is not understanding) that I can’t help but view this as being more about optics than its actual content; i.e. he seems to be cleverer than I am in recognizing most people will not see the letter as a push to use AI in ways which are healthier for human flourishing but as advocating for one of two sides in a highly artificial binary (“pro AI” or “anti AI”).
Solving a theorem is like climbing a new mountain. The mountain is already there, and there is a list of the hardest known mountains to climb. The point of climbing them and not just dropping with a plane on top of the peak is to help develop human climbing skills and expand our knowledge and abilities. Also, already conquered mountains are climbed all the time to test new strategies.
Now, if an LLM proves a theorem, it's like discovering a new mountain and knowing what its peak looks like. Does that mean the problem is finished? No, we still need climbers to actually do the work and advance the field with human understanding.
This is really a microcosm of one particular problem that AI presents to the world: what do people do when their labour is not required any longer?
Here the worry is the social structures of mathematics are eroded such that fewer humans become able to to the work and less well, similarly to how juniors are being recruited less in software engineering, breaking the ladder and leading to fewer seniors in the years to come.
The answer to this really depends strongly on what AI can actually accomplish, but I’ll assume the maximal case and say that AI can do everything economically necessary, and further even those things just desired, such that human labour isn’t required to anything that anyone wants in a practical sense.
Here, we don’t need a human understanding of mathematics to give people a perfect standard of living. We also don’t need humans involved with anything else.
Everything therefore becomes a hobby or a game. People do things because they enjoy them for their own sake, or because they are endeavours used as vehicles to socialise and enjoy others’ company, or because a shared social belief exists and is cultivated such that doing such and such a thing confers social status.
And I think that’s more or less it. I predict we may see some fairly strange sorts of things, such as games where the team structure looks like the descendant of a company org and they compete in an artificial economy. Likewise we may see gamified versions of universities and academia. All of these would be “tamed” such that the rougher parts of the experiences were sanded off.
Sort of like how we evolved in an ancestral environment, and we have certain drives and expectations driven by that environment even though they no longer matter for survival. Our social structures may be derived similarly from those of today, even after they have ceased to serve a real purpose, but changed and repurposed to give meaning and community.