The benchmarking problem

AI companies use benchmarks to prove their models work, but many benchmarks are outdated and companies have learned to game them. Vals wants to fix this broken system.

Meet Vals and its founder

Vals was founded in 2024 by 25-year-old Rayan Krishnan, who previously worked at Palantir, Microsoft, and Stanford's AI lab. He started the company because academic benchmarks weren't keeping pace with AI advances.

How Vals is different

Instead of public tests companies can train against, Vals keeps its benchmarks secret and measures whether models can actually do real work in specific industries like law, finance, and coding.

“Can they do work that produces a product of the same quality as a human within every domain?”

Testing for real-world impact

Vals evaluates both positive outcomes and potential harms, checking whether models could cause problems if deployed widely. The company tests capabilities in areas like cybersecurity, biosecurity, and law of armed conflict.

A business model that works

Companies pay Vals to test their models, similar to how students pay for the SAT. These evaluations help companies improve and are becoming key factors in AI acquisition decisions.

Rapid growth and expansion

Vals raised forty million dollars in Series A funding led by Andreessen Horowitz last month. Revenue is eight times higher than last year, and the team has grown from eight people to twenty-five.

New benchmarks on the horizon

Vals recently launched a program providing model evaluations to federal agencies and plans to expand its team by ten to fifteen more people while moving to a larger office.

The future of AI accountability

As AI companies go public, Krishnan believes benchmarks like Vals will become central to how they report performance to investors and regulators, shaping the entire industry's approach to trust.

“I think as AI models become a core part of the economy, the types of benchmarks and evaluations that we do are going to drive their usage.”