#benchmarks
7 stories
AI still can't do the research that builds better AI
Princeton researchers gave AI agents six days, $3,000, and a GPU budget to write a conference paper. The authors rejected both.
MIT Technology Review · 2m
AI coding agents keep failing on real company code
A new benchmark ran eight frontier models against real tasks from private production codebases, and the best one solved under 40%.
Withspecific · 4m
118 million queries per second
PlanetScale pushed its new Neki database to 512 shards and watched throughput scale almost perfectly linearly.
Planetscale · 1m · 7 views
Coding is solved, but the slop isn't
An Earendil researcher tried to measure how sloppy AI-written code really is, and found agents produce twice the mess humans do.
Earendil · 2m · 6 views
RTK saves tokens, not money
A $1,500 benchmark on Terminal-Bench 2.1 shows the most popular AI-cost-cutting tool barely moves the bill, and sometimes raises it.
Quesma · 2m
SWE-2: near-frontier coding at a quarter the price
Cognition's new coding model scores within a point of Fable 5.1 while costing 64% less, and the trick is how they penalized cost during RL.
Cognition · 3m · 7 views
GPT-6 Astra: OpenAI's Next Frontier Model
GPT-6 Astra introduces major breakthroughs in computer use, software engineering, science, and alignment.
OpenAI · 3m