From RuntimeWire · RuntimeWire · · 30s
Sherpa Labs launches finance coding AI benchmarks
Sherpa Labs releases AI benchmarks for finance deals and production code.
In brief
Sherpa Labs releases AI benchmarks for finance deals and production code. Sherpa Labs launched AI benchmarks testing agents on finance deals and production code tasks. Originally reported by RuntimeWire.
Sherpa Labs launches AI benchmarks
Sherpa Labs introduced finance and coding benchmarks for AI agents on October 20th.
“On October 20th, Justin Jiang (@jostinjiang) introduced Sherpa Labs' finance and coding benchmarks for AI agents, tests built around completing professional work rather than answering isolated questions.”
Benchmarks test professional workflows
The benchmarks measure agents on tasks from private-equity deals and production software repositories.
“Jiang says the company is building reinforcement-learning environments for finance, coding and quantitative research, with the first two benchmarks reporting low pass rates on tasks drawn from private-equity deal work and a production software repository.”
Low pass rates reported
Early benchmarks show low pass rates for agents completing finance and coding tasks.
“Its early scores illustrate the gap the company hopes labs will pay it to close.”
The essence
Sherpa Labs launched AI benchmarks testing agents on finance deals and production code tasks.
