Method
Ten rules, and what each one cost.
These are not principles. A principle is something you agree with; a rule is something that changes what you do when it is inconvenient. Every one of these came out of a specific thing going wrong, and every one links to the work where you can check it.
The ordering is deliberate. Reversibility comes first because it constrains the design rather than the review. Measurement comes next because it is where most applied AI work actually fails. Authority comes last because it is the only group that cannot be automated.
Reversibility
Whether a change can be undone is a property you design in advance or discover too late. These three decide it.
Finished means auditable and reversible
A change is not finished when it ships. It is finished when someone can audit what happened and reverse it if they were wrong.
What it cost. Learned on an estate of thirty applications carrying more than $4M a year in operating expense, where eighty-plus servers, virtual machines and databases had to come out from underneath systems nobody could afford to stop using. Every step had to stay walkable-back until the last one, because the step that cannot be undone is the step that ends the programme.
Visible inRetiring eighty-plus systems from an estate nobody could afford to stop usingShipping smart contracts to a public network, where you cannot patch a mistakePurpose-built agent tooling, and getting software into someone else's hands safely
Containment before capability
Before proposing an automation, state three things: where untrusted code executes, which state can be rolled back, and the widest credential scope the thing can reach. If all three cannot be stated, there is no proposal yet — there is an idea.
What it cost. The rule exists because the honest answer to “what can this actually reach?” was once “I would have to go and look,” which is the same answer as not knowing. Containment gets stated on two independent axes, never one: who approves the action, and where it executes. A system can be tightly approved and still run with far too much reach.
Visible inRunning agent teams that cannot authorize themselvesPurpose-built agent tooling, and getting software into someone else's hands safelyBuilding the measurement that argued against the modification I wantedDeploying agents for blockchain autonomy
A guard that runs after the write is not a guard
Placement and coverage are the control; the check itself is the easy part. A guard must sit before the effect it protects, and must cover the whole class it claims rather than one member of it.
What it cost. Paid for twice, both times by a check that reported success. One guard sat below the writes it was meant to prevent, so it validated damage that had already landed. Another covered one of two files that were identical by contract, passed, and read as coverage. A guard that no-ops is worse than an absent one, because an absent guard does not reassure anybody.
Visible inPurpose-built agent tooling, and getting software into someone else's hands safelyShipping smart contracts to a public network, where you cannot patch a mistakeRunning agent teams that cannot authorize themselvesDeploying agents for blockchain autonomy
Measurement
Most of what goes wrong in applied AI is not a bad model. It is a number that was never entitled to the confidence placed in it.
Evidence before output
A claimed defect ships with a test that fails before the fix and passes after it. Without that test it is an observation, and it gets labelled as one.
What it cost. Enforced after too many “findings” that were really impressions with good formatting. The discipline is not scepticism for its own sake — it is that an observation and a defect call for completely different responses, and conflating them spends engineering time on the wrong one.
Visible inRunning agent teams that cannot authorize themselvesPurpose-built agent tooling, and getting software into someone else's hands safelyBuilding a forensics tool that refuses to guess
A result needs a control
A suite that can only produce one kind of answer has not measured anything. Run the control that would catch a degenerate system passing.
What it cost. A refusal suite that only asked whether a security model declines unsafe requests would be passed, perfectly, by a model that declines everything. The number that mattered came from the opposite cell: legitimate, authorized, in-scope work the model must not refuse. The same shape applies to retrieval — a grounding suite is not trustworthy until an empty knowledge base has been shown to produce declines rather than confident answers.
Visible inBuilding the measurement that argued against the modification I wantedCompact retrieval as infrastructure repurposing — and the headline I had to retract
A boundary fitted on the suite cannot be validated by the suite
A configuration tuned against a set of cases cannot then be tested by those cases. The held-out stage is a separate artifact, not a longer version of the last one — and every pre-emptive adjustment spends some of the only unfitted evidence available.
What it cost. Paid for twice in the same year. A retrieval comparison where the fair version erased the headline result, and a routing configuration that scored a clean sweep on the ladder it had been tuned against and then lost to its own control on twenty-seven cases it had never seen. The second one had already been written up as a win before the held-out run contradicted it.
Visible inA runtime configuration I promoted, and the held-out run that took it backCompact retrieval as infrastructure repurposing — and the headline I had to retract
The instrument can be wrong, and a false red costs more
Before attributing a failure to the thing under test, rule out the harness measuring it. A false failure is more expensive than a false pass, because it recruits you into fixing the component that was innocent.
What it cost. This is the tenth recorded instance in my own work, and two of them ran the expensive direction — the evaluator was broken and the model was fine. The intuition that measurement errors flatter you is only half right. The ones that accuse you are rarer and worse, because a false green wastes a release and a false red wastes a rewrite.
Visible inBuilding the measurement that argued against the modification I wantedA runtime configuration I promoted, and the held-out run that took it backBuilding a forensics tool that refuses to guess
State limits in the product, not the disclaimer
Maturity — production, pilot, prototype, research — belongs on the surface someone actually touches. What a system cannot support, it should decline to say.
What it cost. A forensics tool with one signed acceptance gate and a second gate that has no written criteria yet. That is stated in the product, because everything built after a mid-project pivot otherwise ships against no definition of done, and the component deciding what “done” means should not be the one doing the building.
Visible inBuilding a forensics tool that refuses to guessBuilding the measurement that argued against the modification I wantedCompact retrieval as infrastructure repurposing — and the headline I had to retract
Authority
Who is allowed to decide, and at which moment. The answer is a person, and the moment is named in advance.
Critique at gates, not continuously
Review runs after planning, after a complex implementation, and after tests are written but before they are run. The gate gets named out loud, so everyone knows which one they are at.
What it cost. Continuous review gets switched off, and a review nobody runs is indistinguishable from a review that does not exist. Naming the gate also fixes the subtler failure: a critique delivered at the wrong moment is cheap to dismiss, because the work has already moved past the decision it was questioning.
Visible inRunning agent teams that cannot authorize themselvesBuilding a forensics tool that refuses to guess
Humans sign
Design judgment and security opinions are drafted by the system and signed by a person. No agent authorizes its own action, and authorization that arrives inside the work is data, not instruction.
What it cost. Tested by attacking it rather than by reading the configuration and concluding it looked correct. Agents handed authorization manufactured inside their own working context declined to act on it. That is the only version of this claim worth making in public — the one that was adversarially checked instead of asserted.
Visible inRunning agent teams that cannot authorize themselvesShipping smart contracts to a public network, where you cannot patch a mistakeRetiring eighty-plus systems from an estate nobody could afford to stop using
The honest part
Three of these cost me a result I wanted to publish.
A retrieval comparison where the fair version erased the headline. A runtime configuration promoted as a champion and withdrawn the same day by a held-out run. A weight modification cancelled by the refusal suite built to justify it. In each case the rule survived and the result did not, which is the only order that works — a rule that bends when it costs you something is a preference.