# Coding is solved, but the slop isn't > An Earendil researcher tried to measure how sloppy AI-written code really is, and found agents produce twice the mess humans do. - Title: Coding is solved, but the slop isn't · Earendil | Shortify - Summary: An Earendil researcher tried to measure how sloppy AI-written code really is, and found agents produce twice the mess humans do. Correct code and good code… - Keywords: ai coding, code quality, benchmarks, software engineering, llm evaluation, technology, startups, Coding, solved, slop, isn't, Earendil - Source: Earendil — https://earendil.com/posts/measuring-code-sloppiness - Published: 2026-09-10T00:00:00+00:00 - Read time: 2 min - Topics: ai coding, code quality, benchmarks, software engineering, llm evaluation, technology, startups ## Correct is not good LLMs now write formally correct code almost every time. That same code still piles on needless abstractions, duplicates, and bad structural calls. > Just because the code is formally correct doesn't mean that it is not introducing unnecessary abstractions, creating duplicates, or just making bad decisions overall. ## Features become line explosions Anyone who has vibe-coded a project knows the pattern. Each new feature can blow up the line count far beyond what the feature needed. ## Humans lose the thread When a project adds millions of lines a month, no person can keep up. Agency drains away, and the usual reassurance is that the agents will handle it. > I have bad news for you, agents can't really deal with the slop either. ## The industry runs on vibes Sebastian came from physics expecting measurement. He found marketing lines like "AI that doesn't just suggest code, it ships it" and very little rigor. ## Why correctness was easy Code is verifiable. Generate it, run hidden tests, get a clean reward signal. Sloppiness has no such signal, it needs human intuition and taste. ## AI judges barely work Asking a model to rate code 1 to 10 is close to a random number generator. Show it two solutions and it can flip its preference when you swap the names. > Asking LLMs to judge the code they write is not a substitute for a proper evaluation. ## Humans don't scale A good engineer reviewing the output is the gold standard for keeping code readable. It cannot power training runs or benchmarks across many models and harnesses. > I wouldn't want to force anyone to review millions of LOC just to get an ever changing ranking of model providers. ## Counting lines works, annoyingly Change in lines of code turned out to be a surprisingly good slop signal. Start optimizing for it and it stops meaning anything. > Measuring programming progress by lines of code is like measuring aircraft building progress by weight. ## Verbosity as a metric From SlopCodeBench: take the lines flagged by handcrafted AST-Grep rules plus cloned lines, divided by total lines. Even the rules are human taste in disguise. ## Erosion as a metric Erosion asks how much of a codebase sits in huge, tangled functions. Weight each function by complexity times the root of its length, then take the share above complexity 10. ## Agents score twice as bad Established repos average 0.15 verbosity and 0.31 erosion. Agent code hits 0.33 and 0.68. His own vibe-coded projects reached 0.4 and 0.75. ## Zero percent over time SlopCodeBench wipes model context between rounds of instructions and tests, like real iterative work. Bad decisions compound, and no model passes every checkpoint. > even state of the art models achieve 0% pass rate ## Taste stays in the loop Every workable measure of slop smuggles human judgment inside it. Next up for him: function coupling, code churn, and cohesion. > human intuition and taste are still either implicitly or explicitly baked into the evaluation ## Key takeaway Correct code and good code are different things, and nobody has a reliable automated way to tell them apart yet.