Earlier this year I argued that summarisation is a test of comprehension, not intelligence. It is a filter rather than a crown: a bad summary is strong evidence that someone did not understand the source, while a good one proves less than people want it to. I still think that is right as a claim about what a summary proves about its author. This post is about a different question, where a system should spend its judgment, and on that question the same fact pulls the other way from how the pipelines I have seen are built.
If summarisation is a comprehension test, then every system that summarises has placed a comprehension test somewhere in its pipeline, and the interesting question is who is sitting it. A fixed routine does not fail the test so much as skip it, which is worse, because its output looks like an answer. In the agent setups I have seen, including my own, the answer is usually the cheapest component available. The context window gets compacted by a fixed routine. The daily brief is assembled by a template. The session handoff is a rolling note with a line budget. The test report is read through the last thirty lines because the whole thing would not fit. Summarisation is treated as plumbing, the step you hand to the smallest model or the dumbest script because the real work happened upstream. That gets the shape of the risk exactly backwards.
Here is what the failures actually look like, from my own records over the last few months. None of them failed by being too long.
A capture pipeline that pulls claims out of long source documents preserved the substantive sentence three times running and lost the condition that bounded it. A finding that held only when safeguards had been deliberately switched off became a finding about the system. A regulator’s “encouraged” became “must”. Verbatim quote checking did not catch any of them, because the qualifier sat in a different sentence from the quotable claim. The qualifier reads as hedging, as throat-clearing, as scope boilerplate, so it is precisely what compression discards, and in a regulatory or incident source it is the operative half.
A rolling handoff note with a hard line budget took careful, hedged triage lines and carried them forward as bold unqualified claims, and the bold version then travelled across sessions as if it had been the original framing. Nothing in the compression step was wrong about the topic or the headline number. It stripped the attribution caveat to fit the budget, and without an explicit check against the original line, the gist silently replaced the qualifier.
A test report piped through the tail of a log kept the end of a list of node names and dropped the heading that said the list was of regressions. The surviving fragment had correct indentation, a section, and a pass line. Nothing in it said it was partial. It produced three different wrong readings in one session. The defect worth keeping is not the truncation; it is that a truncated report is indistinguishable from a complete one, because the thing that told you what the list meant was a single line that looked like furniture.
A session-start carry-forward carried work as overdue and top priority when it had already shipped, because the summary preserved the shape of an urgent open item and dropped the fact that the item had closed. And a capture report ended with the right visible label, “read still owed”, while skipping the disk write that would have made the debt survive the session. It kept the appearance of completeness and lost the one durable action.
The pattern is the same in every case I have, and I would expect it wherever the source is careful and the compressor is not. What gets dropped is the bounding condition, the heading, the closure, the action line: the parts that look like structure rather than content. What gets kept is the confident assertion, the number, the list, the label. A summary produced by something without judgment keeps what looks important and drops what is important, and in a careful document the two are frequently opposites, because the load-bearing sentence is often the unglamorous one.
So the consequence of the earlier post is not that summarisation is beneath the intelligent part of the system. It is that summarisation is where the intelligent part of the system should be spent. The model that does the reasoning upstream can afford to be fallible, because its output will be read. The thing that decides what survives into the next context, the next session, the next reader, is making a decision that is rarely re-examined, because what it drops is gone from the next reader’s view unless someone goes back to the source, and nobody goes back to the source of a summary that looks complete. If you have one strong model and one weak one, the weak one should draft and the strong one should compress, and if you cannot afford that, the compression step should at least be told what it is not allowed to lose: every qualifier attached to a claim, every heading that types a list, every closed item, every action. The question to put to any summariser, human or machine, is not “is this shorter” but “what did you decide was boilerplate, and were you right”.