Method
The “O-18” method
“O-18” is 18 procedures for taking a decision apart, distributed across different members of the panel. They are handed out by name: 4 mandatory ones go to EVERY critic, the other 14 go out one by one — each to the critic whose role fits it best, and never twice. That is why critics look at the decision with different instruments instead of paraphrasing one another. A lens counts as applied only if it produced a result, or if the reason it did not fit the case is stated.
What these procedures are
All 18 are below. What we do not publish is the rest of the machinery: how lenses are distributed across roles, the wording of the tasks given to critics, and the order of synthesis — that is precisely the difference between a hearing and a chat prompt.
| Procedure | What it does | Who gets it |
|---|---|---|
| Steelman | first state the target position more strongly than the customer did; attacking a weak version does not count | every critic |
| Pre-mortem (Klein) | a year on, the decision has failed — write the story of that failure from the future, not a list of what might go wrong | every critic |
| Key assumptions | write out EVERY load-bearing assumption, including the unspoken ones, and what follows if each is false | every critic |
| Falsification (Popper) | state plainly the observation that would refute the position, and the cheapest way to obtain it | one of the critics |
| Toulmin warrant attack | break the argument into data → warrant → claim; attack the warrant, not the data and not the conclusion | one of the critics |
| Inversion (Munger) | invert the conclusion: what must be true for the decision to be right; test every such condition | one of the critics |
| Quality of information | walk the facts: source, age, interest, verifiability; flag what is unverified | one of the critics |
| What If? (the surprise has already happened) | take what the target calls impossible as a fact and explain how it came about | one of the critics |
| High impact, low probability | name 2–4 low-probability, devastating events and assess readiness for each | one of the critics |
| Good faith of the source | if the data comes from an interested party — what exactly it gains by showing this and hiding that | one of the critics |
| Competing hypotheses (ACH, Heuer) | list the alternative explanations and run the evidence through a matrix: which hypothesis survives the most attempts to refute it | one of the critics |
| Four points of view (stakeholders) | for each key player: their goal, their version of events, what they will do in response to this decision | one of the critics |
| Failure mode map (FMEA) | for a plan or process: failure modes × probability × severity × detectability; what covers the costliest ones | one of the critics |
| Bow-tie (ISO 31010) | on the principal risk: causes → event → consequences, preventive and mitigating barriers, and the gaps between them | one of the critics |
| Base rate and the outside view (Tetlock) | for any estimate or forecast: what is the base rate in a comparable class of cases, and why this case is not the exception | one of the critics |
| Scan for biases in the position | confirmation, anchoring, sunk cost, survivorship, motivated reasoning — each with the quote where it shows | one of the critics |
| Chain-of-Verification | put verification questions to your own key claims and answer them separately; on divergence, drop the claim | every critic |
| Code and engineering | edge inputs, races, boundaries, dependency failures; what breaks under load and in a partial outage | one of the critics |
This is also where the honest difference between plans comes from, and we prefer to state it as a number. One procedure is one critic's work: the fewer critics, the fewer procedures physically fit into a hearing. A panel of 3 covers 16 of 18; a panel of 7 covers all 18. Whatever went to nobody, we see in our own coverage ledger and know by name; the report states which procedures were applied and with what result.
Where it comes from
The method is not invented: it is assembled from practices decades old, used where a mistake is expensive — in intelligence analysis, in risk management, in the philosophy of science.
- Gary Klein, Performing a Project Premortem, Harvard Business Review, 2007 — source. The pre-mortem: the team assumes the decision has already failed and writes the story of that failure.
- Richards Heuer, Psychology of Intelligence Analysis, CIA, 1999 — source. Analysis of competing hypotheses: every version is tested, not the favorite one.
- ODNI, ICD 203 “Analytic Standards”, US intelligence community standard — source. Calibrated probability language: “likely” carries a number, not a mood.
- Stephen Toulmin, The Uses of Argument, Cambridge University Press, 1958 — source. Breaking an argument into claim, grounds and qualifiers — you see where it rests on nothing.
- Karl Popper, The Logic of Scientific Discovery, 1934 — source. Falsifiability: a hypothesis is worth something when you can say what would refute it.
- ISO/IEC, 31010 “Risk management — risk assessment techniques”, 2019 — source. The risk map and the bow-tie on the principal risk.
A permanent member — the decision-bias reviewer
He is the only one who looks not at the decision but at how you arrived at it: where the reasoning started, what you treated as obvious, where the decision was made before the arguments, and who may have framed it for you — through urgency, scarcity or authority. He sits in on the Full panel plan and above, and votes like everyone else.
The wording stays honest: we write “your words show signs of such and such a bias — here is the quote”, not “we measured your level of bias”. Measuring a person takes standardized tasks, which we do not hand out mid-hearing — and we do not pretend otherwise.
How it shows up in the report
Right after the verdict comes “The essentials” — the three results a hearing owes you regardless of plan: the three most dangerous assumptions your decision rests on (and what backs each), the cheapest check to run before the irreversible step, and the condition under which the decision stops being right. If any of the three is missing, that is a defect: the service detects it itself by checking the issued text, not the intention of a prompt.
Where the debate is part of the plan, its transcript is followed by a map of disagreement: where the panel agrees, where it splits, which fact would settle the dispute, and who changed position under an argument. A few lines that explain what the argument was for.
Beyond that, every report states the panel and why each member was invited, and the lens coverage: which procedures were applied and with what result. Before printing, the document passes two checks — a fact review by another vendor's engine and an editorial pass. A fact taken from the web enters the report only with a link to the primary source; “not confirmed” is better than a pretty invention.
If the client arrived with a situation and had no decision yet, the reports gain a “THE REPORT” section: every draft decision tabled, marking the one accepted and the ones the client rejected. The rejected drafts stay in the document on purpose — six months later, “why didn't we take the second route” is answered by the report rather than by memory.
Where this method is weaker than we would like it to be has a page of its own: where Opponentura gets it wrong.