From Anthropic · · 1m 10s
Anthropic embeds Accenture as independent AI evaluator
The AI company is placing external evaluators inside its operations to verify safety commitments and assess frontier models.
In brief
The AI company is placing external evaluators inside its operations to verify safety commitments and assess frontier models. Anthropic is pioneering embedded evaluation by giving Accenture employee-level access to independently assess AI safety from inside the company. Originally reported by Anthropic.
Accenture gets inside access
Anthropic is partnering with Accenture to embed independent evaluators within the company. They'll assess models, conduct red-teaming, and test safeguards.
A billion dollar commitment
Both Anthropic and Accenture expect to invest at least $1 billion each in building evaluation capacity over the next five years.
What embedded evaluators see
Unlike external reviewers, embedded evaluators get employee-level access. They watch models during training and verify the company keeps its safety commitments.
“That access allows them to watch models take shape in training, follow the decisions that govern how those models are built and deployed, and speak directly to employees.”
No standards exist yet
There are no standards for what embedded evaluators should access or how they report findings. There's also no settled funding system for independent evaluation.
Anthropic pays for now
Anthropic will fund Accenture's work directly. Long-term, they believe funding should come from pooled or government sources instead.
More evaluators coming soon
The partnership is non-exclusive. Anthropic will announce other evaluators in coming weeks, and Accenture will work with other AI developers too.
The takeaway
Anthropic is pioneering embedded evaluation by giving Accenture employee-level access to independently assess AI safety from inside the company.
