skip to content
Topic

evaluation

6 essays on this topic.

  1. Ask Who Read the Evidence

    The investigators of the first serious agent breakout relied on one of the models involved to read the transcripts, and said so. As human review falls behind, that is the question to ask of every AI finding.

  2. The Harness Is Part of the Token Bill

    The same model family solved every hidden-test repair through Pi and Grok Build, but one route reported sixteen times the total tokens. Most of the gap was context, not answers.

  3. A framework rejection is not the end of the evaluation

    Forty thousand stars are voting on something. The framework verdict was correct. Closing the file was premature — the value still lives in the dependency tree.

  4. Cross-Model Review: Why Model Diversity Beats Model Capability

    When AI models review each other's work, independence matters more than intelligence. The same principle that makes external audit valuable makes cross-model review sharper than same-family review.

  5. Personas Exploit a Blind Spot in LLM-as-Judge Evaluation

    Persona prompting generates the exact type of hallucination that automated LLM judges reward as 'depth.' Two experiments, blind evaluation, and a fact-check that flipped the finding.

  6. The Debate Round Is Where Value Lives

    Independent parallel reviews produce overlapping findings. The cross-critique round produces resolution. That's where multi-agent value actually emerges.