OpenAI's pattern of chaos

OpenAI has repeatedly made impressive mathematical breakthroughs then bungled their announcements, alienating the mathematical community it now seeks to repair relations with.

A new advisory group forms

On September twenty-first, nine elite mathematicians announced the formation of the Advisory Group on Mathematics and Artificial Intelligence, or AGMAI, in a post on UCLA professor Terence Tao's blog.

What AGMAI is supposed to do

The group will operate independently from OpenAI and advise it and other frontier AI labs on the review and communication of emerging results, with members free to challenge the company publicly.

Confusion over independence

Many mathematicians assumed AGMAI was OpenAI-appointed because OpenAI's louder rollout drowned out the group's own announcement, and the group's website revealed OpenAI had approached some eventual members first.

Martin Hairer defends independence

AGMAI member Martin Hairer, a mathematics professor at Imperial College London and EPFL, stressed the group receives no financial, technical, or other support from OpenAI and members have signed only standard confidentiality agreements.

“We don't work for OpenAI, are not paid by them, and it's totally independent.”

Hairer criticizes OpenAI's announcement

Hairer acknowledged OpenAI's announcement did little to help AGMAI establish independence and said the company's careful wording and strong PR skills made it easy to misinterpret the group's relationship with the company.

“The way they phrased their announcement didn't really help.”

The group formed in a rush

Hairer said AGMAI started just two days before his interview with The Verge, suggesting the group came together hastily without a detailed master plan for its work ahead.

“So we, well, you know, started two days ago. So it's not like we have a big master plan.”

OpenAI's massive backlog

OpenAI says its unreleased internal model has resolved more than one hundred long-standing open problems across most areas of mathematics since solving the Navier-Stokes Millennium Prize problem.

OpenAI behaves like an outsider

University of Connecticut mathematics professor Álvaro Lozano-Robledo said no academic announces they have proved many things but do not know what to do with them, calling OpenAI's behavior fundamentally unlike academic practice.

“This is not how any academic behaves.”

Poor scholarship and secrecy

Researchers complained of poorly written manuscripts, scant engagement with existing literature, and OpenAI quietly changing documents after release without announcing alterations, which Hairer described as shoddy and sloppy scholarship.

Solutions without understanding

Lozano-Robledo said the burden on mathematicians is that OpenAI produces solutions without the understanding that traditionally accompanies them, and AI companies misunderstand that solutions alone are not enough.

“Solutions are often less important than the understanding that comes with them.”

Researchers fear disruption

OpenAI hints at solving another Millennium Prize problem and has more than one hundred results ready to release, creating dread among researchers that their years or decades of work could be abruptly disrupted.

Uncertainty paralyzes researchers

University of St Andrews mathematics professor Colva Roney-Dougal said the uncertainty about what OpenAI will announce leaves her unclear whether to rush papers out, wait passively, or continue working normally.

“It's somewhat agonizing to know that a 'large number' of results are likely to be announced soon.”

Hairer's commitment to the field

Hairer said he joined AGMAI partly to address the severely misaligned goals of AI companies and the mathematical community, and he is willing to let OpenAI spin his involvement if it protects the broader math community.

“They're not going to sort of damage me within the math community. In some sense, I don't really care about what they say. But what I do worry about is the math community as a whole.”