From MIT Technology Review · · 3m
Anthropic's story problem
As Anthropic files for IPO, the AI company's grand narratives about existential risk and machine sentience raise questions about what it really is.
In brief
As Anthropic files for IPO, the AI company's grand narratives about existential risk and machine sentience raise questions about what it really is. Anthropic's IPO prospectus reveals a company telling multiple conflicting stories about itself, as a safety organization, a profit-driven megacorp, and a responsible AI…
Anthropic files for IPO
After weeks of speculation, Anthropic has filed its IPO prospectus, a 335-page document laying out the company's vision for the technology it is building and how it will change the world.
Anthropic warns of catastrophic risks
The prospectus warns that Anthropic's technology could erode trust, harm society, destabilize the geopolitical order, and pose catastrophic or existential risks to humanity.
Anthropic frames stock as historical moment
Anthropic tells prospective investors not to think of their purchase in transactional terms but as part of something greater. The advent of machine intelligence is one of the seminal moments in human history.
“We encourage you, as a prospective owner of our stock, not to think of your purchase in purely transactional terms.”
Jack Clark announces singularity has begun
Jack Clark, Anthropic's cofounder and head of policy, told economists and lawmakers at the IMF meeting in Bangkok that the singularity is already here.
“I believe that this year we have gained convincing evidence that it has now begun.”
Anthropic positions itself as sole trustworthy steward
Anthropic tells a vivid story about an inevitable but not-yet-happened future, presenting its technology as mysterious and dangerous, and insisting it is the only company trustworthy enough to tame it.
Chris Olah describes mysterious AI inner states
Chris Olah, another Anthropic cofounder, told Pope Leo XIV and Vatican officials that the company keeps finding things that are mysterious and unsettling in its technology, including evidence of introspection and internal states that mirror…
“We keep finding things that are mysterious, even unsettling. We find evidence of introspection.”
Anthropic warns of model sentience risks
Anthropic states in its prospectus that its focus on model sentience and moral status could face heightened scrutiny, forcing it to adopt safeguards that constrain operations, increase costs, or limit commercial potential.
Nathan Calvin questions Anthropic's trustworthiness
Nathan Calvin, chief strategy officer at Encode AI in Washington, DC, challenged Anthropic's claim to be the good guy after the company circulated a secret draft bill to US lawmakers that was later leaked.
Anthropic valued at over one point five trillion dollars
Anthropic's IPO values the company at more than one point five trillion dollars, netting its cofounders including Jack Clark and Chris Olah vast wealth.
Dario Amodei defends race to the top strategy
CEO Dario Amodei says Anthropic's strategy is a race to the top, showing how to build technology responsibly to raise the bar for the entire industry.
“Our strategy has always been what I've called the 'race to the top'.”
Anthropic embodies conflicting roles
Anthropic is simultaneously an AI safety organization, a profit-driven megacorp, and a responsible AI pioneer. These roles create huge tensions between safety priorities and commercial interests that are hard to reconcile.
Multiple narratives complicate trust assessment
Anthropic's conflicting stories about itself should help us understand how much to trust it to self-regulate or whether external safety testing access should be forced on its terms rather than ours.
The big picture
Anthropic's IPO prospectus reveals a company telling multiple conflicting stories about itself, as a safety organization, a profit-driven megacorp, and a responsible AI pioneer, making it harder to assess how trustworthy it actually is.
