Production Feedback Loops: Turning Agent Failures Into Verified Harness Changes Without Retraining

Agent systems can be improved without touching model weights, and the published gains are large. Teams edit a system prompt, tighten a tool schema, add a skill, change what context is retrieved, or fix a grader. Published measurements show how much room there is: on SWE-bench Verified, switching the scaffold around the same model made what Epoch AI calls "up to an 11% difference for GPT-5 and up to a 15% difference for Kimi K2 Thinking", and on a 112-task benchmark of long development sequences, holding gpt-5.4 fixed and changing only the agent harness produced resolve rates from 7.14 to 18.75 percent, 8 against 21 tasks in single trials. What the same measurements rarely show is which part of the harness produced the difference. Scaffold comparisons swap whole products at once.

Research library