Quit or stay? The AI safety worker's dilemma

AI staff who think their work could end the world are split on whether leaving is principled or just clearing the room.

6 cards · 1m 30s · tap to begin

From · · · 1m 30s

Quit or stay? The AI safety worker's dilemma

AI staff who think their work could end the world are split on whether leaving is principled or just clearing the room.

In brief

AI staff who think their work could end the world are split on whether leaving is principled or just clearing the room. If the people most worried about AI risk all walk out, the ones left are the least worried, which is exactly the wrong sorting. Originally reported by Business Insider.

A resignation goes public

A resignation goes public

Jacob Coxon, formerly of OpenAI, posted on X that he was leaving Anthropic because his colleagues sincerely think the technology could kill everyone within ten years.

“people building AI earnestly believe that it could kill us all by the end of the decade”

A safety lead agrees

Evan Hubinger, who runs Anthropic's alignment stress-testing team, put the odds of AI killing all humans in the next decade at better than 10%.

Leaving empties the room

Gladstone AI's Jeremie Harris says he hears these fears often from lab staff. His worry: if the worried keep resigning, the labs fill up with people who never worried.

“The people who are left are going to be the ones who are least concerned about safety”

Or you stay and push

Anthropic's Anna Wang replied that she works there because she thinks she can cut risk from inside, while respecting people who choose to do it from outside.

“can do better at reducing risks from the inside”

Insiders rarely win

Stevens Institute professor Alex Wellerstein counters that internal objectors have a weak record against institutional momentum, and staying makes you part of it.

“If you stick around, you're participating”

The Manhattan Project rhyme

Navy undersecretary Ralph Austin Bard resigned weeks before Hiroshima after urging a warning to Japan. Scientists who pushed back were shut out of future defense work.

“Whistleblowers can do tremendous good, but are apt to be discredited or harassed into silence”

The essence

If the people most worried about AI risk all walk out, the ones left are the least worried, which is exactly the wrong sorting.

Read the original on Business Insider

React

Sign in to react and comment.

Comments (0)

Life is short. Keep it sweet. Respect others' opinions and be kind!

    Recommended next

    More decks on ai safety and related topics.