From TechCrunch · Lucas Ropek · · 1m 50s
AI benchmarks need fixing before we trust them
Hugging Face's policy chief says model evaluations are too inconsistent and embedded evaluators won't solve the problem without major standardization.
In brief
Hugging Face's policy chief says model evaluations are too inconsistent and embedded evaluators won't solve the problem without major standardization. Irene Solaiman, Hugging Face's chief policy officer, says AI model benchmarking methods are immature and lack reproducibility, making embedded third-party evaluators an…
AI labs bet on benchmarking
AI companies have staked their reputations on model benchmarking, a supposedly neutral system designed to measure whether one AI product is better than another.
Methods are still maturing
Irene Solaiman, Hugging Face's chief policy officer, said at TechCrunch Disrupt that the methods behind these evaluations are complicated and still maturing. Testing needs to become more consistent science.
Reproducibility remains a problem
Solaiman said reproducibility in model evaluations has long remained one of the stickier aspects of benchmarking. Can you reproduce results and access the model itself?
“In terms of whether we can trust results, one big part [of that] is can you reproduce the results of an evaluation?”
Tests can't be compared
Evaluations are an evolving science where the industry does not know how to meaningfully compare them to each other. Tests start to get ineffective relatively quickly.
“They are an evolving science, where we don't know how to meaningfully compare them to each other.”
Embedded evaluators lack standards
There is hunger for an embedded evaluator ecosystem, but the qualifications that determine who is an effective evaluator across what tasks need much more standardization.
“There's this hunger for an embedded evaluator ecosystem now, but the qualifications that determine who is an effective evaluator across what tasks need a lot more standardization.”
Labs prefer third parties over rules
AI labs have pushed for embedded evaluators, supposedly neutral third-party observers inside AI organizations, instead of active government regulation to identify safety concerns.
Evaluators face credibility questions
Doubts remain over how neutral or effective embedded evaluators will be. Anthropic's decision to hire Accenture as its first evaluator spurred backlash over the consulting firm's qualifications.
Evaluation scope is expanding
Evaluations now test the entire system around a model, including the harness and agent. A weaker model with a better harness could outperform a frontier model. Agentic evals are really hard and expensive.
“Agentic evals are really hard and really expensive.”
The bottom line
Irene Solaiman, Hugging Face's chief policy officer, says AI model benchmarking methods are immature and lack reproducibility, making embedded third-party evaluators an unreliable solution without industry-wide standardization.
