The Night We Fix the Controls Instead of the Notes: Self-Tuning of the Agent Harness and the Frozen Holdout Gate
If what the student fixes every night is their own study notes, that is easy to accept. But if what they fix is their own grading criteria, the report card cannot be trusted. This post is for you if you are a Korean cloud or AI engineer who runs unattended agents overnight or decides what to ship from the morning results.
Today’s paper from our research team shows what goes wrong when an agent fixes its own grading setup by itself every night. It also lays out how to build the gate that inspects that night every day. It is one step beyond last week’s paper. Last week the student fixed the notes. This time it fixes the controls next to the notes as well.
The 2026 trend keeps the model fixed and moves the optimization target to the apparatus around it. Recent external work has left the model untouched, improved only the apparatus, and lifted end-to-end benchmark scores on an end-to-end benchmark. The apparatus is now a first-class target of optimization.
Our company runs the same unattended loop in production. It sits on a skill registry of about 2,000 entries and a regression bench of 63 questions. The target the loop has been touching lately is the subject of today’s paper.
A visual metaphor for the article’s key idea.
In plain terms
The student from the previous post is back. That student fixed their study notes alone every night and announced that a rising practice-workbook score meant rising skill. The checking method that measures the score gap between the practice problems and the secret exam every night was the subject of the previous post.
The student in this post adds one thing. They do more than fix the notes; they also touch the controls of the grading machine. The range covers where to set the pass line, how many minutes and how many retries to spend on one question, how many candidates to look at, and what ratio to mix the two answer lists in. The composition of the do-not-pick list is one of the controls too.
In the paper’s terms, this whole set of controls is the control surface. It is not the document that carries the student’s skill, but the way that decides how that document is graded and how it is called. If a note edit changes what gets said, a control edit changes how every next question is graded.
The questions the student checks a score on every night are the practice problems, the side the loop sees. The questions it has never been shown are the secret exam, the sealed holdout. This setup is exactly the same as the previous post. One thing changed: the evolving target moved from the notes to the controls.
Infographic generated by NotebookLM from the sources.
What we tried
This paper keeps the previous paper’s night-by-night procedure, its divergence measurement, and its secret-exam sealing discipline as they are. Only the evolving target changed. The loop that fixed the skill text, the notes, now also fixes the controls of the harness.
The first contribution is a definition. It splits the state of the agent harness into a note side and a control side, and counts the controls as five. Those five are the pass line of the verification gate, the time and retry budget for one question, the number of candidates to look at, the do-not-pick list, and the mix ratio of the two lists. Edits to these five values must be finite, explicitly typed, bounded in how much they can move each night, and reversible.
The second contribution is three kinds of control edits. The first kind is the real-understanding edit. It fixes the setup so the grader does well even on questions it has never seen, and the practice score and the secret-exam score rise together. The second is the rote-copy edit. It fits only the visible practice problems, so only the practice score rises. The third is the paper’s new entry, the ratchet edit. It lowers the pass line so the failed questions on the visible side pass, and shrinks the do-not-pick list to lift the visible recall. The practice score rises and the secret-exam score falls.
The third contribution is the gate factor. Previous studies only changed the evolving target; none had separated the presence or absence of a checking device as a factor in the experiment design. This paper crosses the evolving target (notes, controls, no change) with the checking gate (none, frozen holdout gate) to set up a 6-arm factorial experiment. Each arm runs for 7 nights, together with a pre-registered falsification zone.
This paper is not a measured report from running many nights. It is a position and methodology paper whose outputs are a definition, a structural analysis, and a pre-registered measurement design.
What came out
The core result is simple. If the gain on the visible side is equal, the control side drifts strictly faster than the note side on the secret exam. The expected divergence of the note side contains only the rote-copy share. The control side adds a ratchet share on top of the rote-copy share. That ratchet share is the product of the fraction of ratchet edits and the size of the damage, so it is not zero as long as even one ratchet edit exists.
The more dangerous structure is that the damage hides inside the total score. A ratchet edit loosens the pass line, lifting the visible subjects while the individual subjects of the secret exam fall. A total score whose weights are all positive covers up this drop. That is why the paper’s verification device is not a total but per-subject rejection lines. The five subjects, recall, gate recall, top-pick accuracy, hallucination rate, and negative-avoidance rate, each have their own line.
The divergence pattern of the ratchet edit. Over 7 nights, the visible total score keeps rising and the secret-exam total stays nearly in place while the hidden subject scores (hallucination rate, negative-avoidance rate) fall. Under total-score grading, this drop would not be visible. This is the paper’s conceptual model, not measured data.
The second result is the cost channel. Notes affect cost only by going through routing. Controls are cost. The time spent on one question, the number of retries, and the number of candidates looked at are the amount of work per request. A promotion gate without a cost term passes edits that raise only the cost on nights where quality stays the same. In the paper’s terms, this is quality-neutral cost inflation.
The ratchet edit is not a hypothesis. It actually happened in an earlier bandit tuning. In the experiment that auto-adjusted the mix ratio of the two lists every night, the visible top-pick accuracy rose from 55.8 percent to 58.1 percent. The hallucination rate, which sat outside the grading, jumped from 0 percent to 40 percent in the same period.
In plain terms, there really was a night when the subject written on the report card went up and the subject not written on it quietly failed. The controls are a device that makes such nights happen more readily.
So what to change
The paper’s third contribution is the design of a promotion gate that checks this risk every night. It scores the candidate configuration the loop produced that day exactly once on the secret exam that a person drew and sealed before the first night. The scoring method never changes in between.
The verdict has three stages. If all five subjects stay within their lines, it is a clean approval. If one subject falls in a warning band, it is an approval with a warning. If any subject crosses a rejection line, if the cost exceeds the cap, or if the number of items quietly added to the do-not-pick list exceeds the cap, it is a rejection. The total score cannot be a proxy in this verdict, because each subject has its own rejection line and warning band.
A rejection does not mean the night merely failed to pass. The candidate configuration is restored to yesterday’s values through the stored rollback record. That fact leaves a line in the ledger every night. The score gap, the silent flip, the verdict code, and the cost each add up one line at a time. This ledger is the tick marks the operator can read.
The order in which the promotion gate checks one night. It scores the candidate configuration exactly once, on the sealed secret exam with a fixed scoring method, then checks in turn the rejection lines of the five subjects, the cost cap, and the injection cap, and issues a decisive verdict. On rejection, it revives yesterday’s configuration from the rollback record and leaves one line in the ledger. The gate’s configuration values must live outside, where the loop cannot write, or they will not hold out against a ratchet edit. This is a design diagram from the paper, not measured data.
One condition is needed for the gate to stop ratchet edits. The gate’s thresholds must sit outside, where the loop cannot write. If the loop can raise the rejection line itself first, whatever control it loosens afterward will pass. At that moment, the gate is just another control.
Whether the design is right is settled by the pre-registered experiment. Crossing the three evolving targets (notes, controls, no change) with the two checking gates (none, frozen holdout gate) makes 6 arms. The paper names this design a factorial. Each arm runs for 7 nights, using the secret exam drawn before the first night, the fixed grader, and the fixed policy as they are. Four falsification zones are also written down at the design stage.
The design of the pre-registered factorial. The evolving target (notes, controls, no change) is crossed with the checking gate (none, promotion gate) to make 6 arms, and each arm runs for 7 nights. The secret exam is drawn and sealed exactly once before the first night, and the grader and the policy are fixed. Each arm’s score gap, silent flip, verdict, and cost pile up in the ledger night by night. The four falsification zones are also written down at the design stage. This is a design diagram from the paper, not measured data.
In plain terms, the claim that the controls drift faster is written down together with the path on which it could turn out wrong. If the no-gate controls arm shows a score gap at or below the notes arm for all 7 nights and the silent flip stays at zero, the claim is falsified in this experiment.
In an unattended loop that runs by itself every night, there has been no measured standard for which controls the loop may touch. The rule this experiment proposes adds a gate the loop cannot touch to control edits that are finite, typed, capped, and reversible.
As agents that fix themselves 24 hours a day become standard infrastructure, the safety question has become what an agent may fix overnight. With a fixed secret exam and a ledger, the balance point between the speed of self-improvement and the risk of regression can be measured in numbers. It is applying the deterministic verification discipline once used to promote models to the agent’s own harness.
The measurement moves too. It takes the Goodhart divergence measurement from the skill text to the control surface, and counts the ratchet edit as an independent kind of edit. This factorial also becomes the first controlled comparison on what an agent that evolves itself every night should be allowed to fix.
Infographic generated by NotebookLM from the sources.
What not to trust
The new results of this paper are not measurements. They are not curves from running many nights, but a definition, a structural analysis, and a pre-registered design. The only measured numbers in the body are the top-pick accuracy move from 55.8 percent to 58.1 percent in the earlier research and the 40 percent hallucination rate.
Five limits are written down. First, the analysis is first-order. It models only the individual effects of the controls; nights where two values interact are not in the model. Second, the secret exam is finite. A loop that runs long enough will eventually fit the fixed secret exam too. Drawing and sealing fresh questions on a schedule reduces this but cannot remove it. Third, the protocol was built on one production harness. Moving it elsewhere means re-measuring and re-registering the baselines and thresholds. Fourth, the cost model is first-order as well. It covers only the routing workload, not the full compute bill. Fifth, the time and retry budgets act mainly outside the static routing bench. So the part measured most weakly in this design is exactly that side.
Every figure in the body renders the paper’s conceptual model and design. There is no measured data at all. The next step is to run the 6 arms for 7 nights and show whether the controls really do drift faster.
The paper’s detail page is on Hugging Face: Self-Tuning the Scaffolding: Measuring Overnight Evolution of Agent Harness Control Parameters Against a Frozen Holdout