Penn State Tested 5 Coding Agents Patch by Patch: 38% to 82% of Successful Runs Were Already Correct on the First Edit

A coding agent usually works out the fix long before it stops typing. That sentence is the finding of a preprint from three researchers at Pennsylvania State University, and the paper gives the gap a name: termination lag, defined as the number of steps between an agent's first correct patch and the end of its run.

The method is what makes the result worth reading. Musa Cim, Burak Topcu and Mahmut Taylan Kandemir did not score final submissions the way a leaderboard does. They recorded every distinct repository patch an agent produced during a run, then tested each one, across five models, two agent frameworks and three SWE-bench datasets. That turns a single pass-or-fail outcome into a timeline of when correctness actually arrived, and their preprint, arXiv 2611.14960v1, was submitted on November 11.

The headline numbers are wide but consistent. In 38% to 82% of successful runs, the agent's initial source-code edit was already a correct solution. Everything after that point is lag. In successful runs on SWE-bench Verified and SWE-bench Multilingual, post-solution steps accounted for a median 35% to 37% of the run for Qwen3-Coder and 54% to 60% for Laguna-XS.2, meaning the agent spent more of its budget after the answer than before it. And in 1% to 8% of cases, the continued activity undid the earlier correct patch and left the final submission incorrect.

This is invisible on the benchmark that the field uses. SWE-bench scores the final submitted patch against task-specific tests, so two agents that both get the repository right at step ten receive identical credit even if one stops there and the other burns forty more steps rerunning tests, inspecting callers and reviewing its own diff. The authors put it plainly: the final score obscures when the correct solution was actually obtained, and that timing is a primary driver of both execution duration and cost.

The extra steps are not random thrash. Most of the post-solution effort goes into verifying the work and exploring the repository, which is exactly the behavior that makes agents useful on unfamiliar codebases and exactly the behavior that is hardest to bound. The uncomfortable corollary is that continued work is not the same as improvement. An agent that keeps going can downgrade a solved task, and the customer pays for the tokens that did it.

Three caveats. The paper is a preprint and has not been through peer review. The abstract names only two models for the median figures and only two of the three datasets, so the full spread across the other models, frameworks and dataset is not established by the summary alone. And the 38% to 82% range is a range across runs and configurations, not a single rate that any one team should expect.

The operational lesson is cheap to act on. If most successful runs are correct several steps before they end, a stopping rule that halts when the test suite passes is a cost intervention that does not require a better model, and it removes the tail risk of an agent breaking its own fix. Teams paying per token should also stop reporting only pass rates: the number that matters commercially is cost per finished task, and termination lag is a direct component of it.

Read alongside the week's other agent news, the paper points at where the engineering effort is moving. Model capability is no longer the obvious constraint on agent economics. The loop around the model, including when to stop it and how to check that it stayed in bounds, is now where the measurable savings sit, and it is the part of the stack that most teams have not instrumented at all.