skip to content

What A Prompt Trim Cannot See


Anthropic removed more than eighty percent of its coding agent’s system prompt for the newest model generation and measured no loss on coding evals. Someone on that team published the reasoning the day the models shipped, and it is a good piece of writing. The old prompt was thick with hard bans written to stop weaker models doing stupid things, and the current generation handles the nuance without being told, so rules give way to judgment. Worked examples narrow the space a model explores, so they give way to more expressive tool parameters. Everything loaded upfront gives way to progressive disclosure. I agree with all of it, and I have been trimming in that direction for months.

Then I applied it properly to my own agent setup and hit a caveat I have not seen anyone state. This advice is safest for the people who never took their instructions seriously, and most dangerous for the people who did.

The first way it goes wrong is that duplication is sometimes the contract rather than the waste. My audit flagged a writing rule that appeared in three places, marked it redundant on the reasoning that the other two copies covered it, and cut it. That was exactly backwards. A test asserted that all three surfaces carry the same rule, deliberately, because the replication is an anti-drift mechanism, and a sibling test asserted that one specific copy is the authority the others defer to. The line I deleted as a duplicate was the source. The test suite caught it as one regression above baseline, and it reproduced on a same-commit rerun so it was not flake, and I put the line back. The lesson generalises past my particular setup. Asking whether a line is duplicated is the wrong question. The question is which copy is authoritative, and whether something in the repo requires the copies to agree.

The second way is worse, because no amount of careful reading finds it. A trim audit reads prose and classifies lines. It never looks at the code that consumes the prose. In my case a function pulled a specific section out of the config file by its heading and injected it into every subagent I spawned, returning early and silently when it found nothing. The heading had been renamed four months earlier. For four months no subagent inherited a single one of those rules, nothing logged a warning, and nothing failed, because an empty return is indistinguishable from having nothing to inject. In the same audit I found a test that had been correctly failing for sixteen days, reporting real violations, into an empty room. A dilution audit measures neither of these. So the file can look tidy while buying you nothing at all.

Those two failures compound. Cutting a line because a hook enforces it is safe only while the hook is live, so the entire argument for deleting prose that the machinery already covers collapses the moment the machinery is dead — silently, by construction. The config that looks best maintained after a trim can be the one enforcing least.

It is worth being precise about what licensed Anthropic’s cut, because it is doing more work than the reasoning around it. They had coding evals showing no loss. That measurement is the permission slip. Without an equivalent you are not applying their result, you are copying their conclusion, and most people reading that post have a config file and a couple of hooks and no evals at all. The reversal they describe is a claim about model capability. It is not a claim that your particular instructions were doing nothing, and only your own measurement can tell you that.

What I do now is cheap enough that I no longer trim without it. Before I cut a line as redundant I grep the tests for its phrasing to see whether one of them requires the duplication, and before I cut a line as superseded by enforcement I grep for the enforcement and confirm it fires. Every string anchor running from code into prose has a test asserting the anchor still resolves, and each of those tests has a negative control, because if renaming the heading does not turn it red I have not tested anything. I run the suite at the start of an audit rather than the end, since a pre-existing failure is evidence about the very rules I am auditing. And when a rule names a path or a directory I check the filesystem for violations myself, because the guard is the thing under suspicion.

None of this argues against the trim. Models did get better, most accumulated instruction is scar tissue, and I cut a lot of mine and would cut it again. But a smarter model licenses deleting instructions. It does not license deleting the machinery that made those instructions true, and it does not tell you which of the two you are looking at. Trim the prose. Test the anchors first.

Related by topic
  1. What Anthropic's Managed Agents validates — and what to steal
  2. The architect-implementer split: why your expensive model shouldn't write code
  3. Building porin: a library for agent-facing CLIs