OpenAI drops nearly 400 results

OpenAI released nearly four hundred AI-generated mathematical results spread across more than seven hundred manuscripts covering combinatorics, geometry, number theory, algebra, topology, and other disciplines.

Mathematicians struggle to process scale

More than three dozen mathematicians told The Verge the flood was staggering, overwhelming, unprecedented, and surreal. Simply understanding what OpenAI released could take years, researchers said.

“Staggering. Overwhelming. Unprecedented. Surreal. Pure insanity.”

Verification is incomplete and uneven

OpenAI acknowledged results are at different stages of verification. Fewer than half the manuscripts have been formally verified in Lean, a programming language that allows computational proof checking.

Kevin Buzzard finds mostly unverified work

Imperial College London mathematician Kevin Buzzard identified only around six theorems in algebraic number theory that stood out, and few appeared formally verified in Lean.

“Hence, I either have to read possibly-not-correct slop, or wait for others to do the same, or wait for someone to formalise them before I can say for sure that the results are even correct.”

AI slop is a widespread problem

Mathematicians have seen a huge uptick in low-quality, erroneous AI-generated material from tools like ChatGPT and Claude. Much is confusing, hard to read, and demonstrates little understanding of the subject.

Papers are poorly written and incomplete

Many mathematicians described papers that were difficult or practically impossible to follow. Some appeared to cover ground already trod by other mathematicians, with unusually brief write-ups.

“The write-up of the problem I know best made little sense after a quick read.”

Some results are genuinely impressive

Despite shortcomings, researchers agreed the release contains genuinely impressive work. Many results would warrant publication in top-tier journals and could secure academic careers for their authors.

Major unsolved problems appear solved

Stanford mathematician Jared Duker Lichtman identified tens of results in major problem areas, including progress toward the Riemann hypothesis, a special case of the Hodge conjecture, and a solution to the four-dimensional Kakeya conjecture.

Research programs are wiped out overnight

Mathematicians reported that years of work and carefully laid research plans evaporated instantly. Colva Roney-Dougal at St Andrews University said friends and colleagues had grant proposals wiped out.

“I have a bunch of friends and colleagues whose grant proposals have just been wiped out.”

Some fields are hit harder than others

Researchers in probability, combinatorics, and theoretical computer science appeared particularly affected. Francesco Fournier-Facio said some regions of group theory had been bulldozed.

OpenAI appears to target specific areas

Scott Armstrong said there were definitely some targets. OpenAI focused on areas where Hong Wang won the Fields Medal this year and several Millennium Prize problems, including Yang-Mills theory.

“There were definitely some targets.”

OpenAI consults advisory group on responsibility

After bruising encounters with researchers earlier this year, OpenAI worked with the Advisory Group on Mathematics and Artificial Intelligence on how to release results responsibly.

OpenAI follows some but not all advice

OpenAI said it would fund workshops and conferences and disclosed more information than before, including that its model attempted more than four thousand problems. But it did not identify the model or disclose prompts used.

OpenAI plans to keep testing on mathematics

OpenAI indicated it has no plans to stop testing its models on mathematics problems, saying it is important to continue evaluating frontier models to accelerate developing tools for those fields.

Years of work lie ahead for mathematicians

Researchers estimated that making sense of everything OpenAI released could take the community years. Scott Armstrong likened it to aliens visiting Earth and leaving behind mathematics to study.

“If the AIs would disappear now, as though there were aliens that came to Earth and then just left, we would be studying this for the next 10 years, trying to understand everything.”

Results use existing methods, not new ones

Jared Duker Lichtman said all the results come out of existing methods and techniques, filling in parts of the known mathematical landscape rather than creating entirely new ones.

Junior researchers face existential uncertainty

The disruption is hitting PhD students and junior researchers particularly hard. Open problems can underpin dissertations, grant proposals, and job applications, all important for academic careers.

Mathematicians fear repeated waves of releases

OpenAI's findings have arrived in increasingly large waves with little pause between them. Rumors are already circulating about further releases, and researchers dread another drop or Anthropic entering the fray.

“There's going to be something in two more months that's going to blow this away. It's just, like, repeated strikes by bigger and bigger bombs.”

Even AI optimists are growing uneasy

Scott Armstrong, who uses AI in his own work and is among the most enthusiastic mathematicians about it, said it is kind of hard to sleep at night and questioned what kind of future there…

“It is kind of hard to sleep at night.”