Methodology in Code

How the Whewell falsification framework runs — which dimensions are code-enforced gates and which are operator protocols.

The Consiliences Institute publishes a methodology page describing its falsification framework. This page is the technical bridge: how those criteria run in practice — which are gates the code enforces, which are protocols the operator follows — and why “killed” is a rule-driven outcome rather than an editorial mood.

§ I   The principle

A research programme that confirms everything is confirming nothing. The platform’s credibility depends on visibly killing its own findings — not occasionally, not editorially, but mechanically, at a published rate. The kill rate is part of the publication itself.

For that to be credible, the kill decision must run on rules that pre-date the finding — not on a judgement call made by someone with an incentive to keep it alive. The falsification battery and the audit gate exist so that the rules, not the mood, decide.

§ II   Whewell, eight dimensions

The Institute’s framework follows William Whewell’s nineteenth-century criteria for genuine consilience, extended for modern statistical practice.

The eight dimensions below are not equal in how they are enforced — and saying so is the honest version of this page. Two are hard code-enforced gates that block output. The rest are a mix: some are recorded fields and warn-level checks, others are protocols the operator follows. Where a dimension is enforced in code, this list says so; where it is a working practice, it says that instead.

• Independent prediction. An agent constructs a forward-out-of-sample test before in-sample work is read; the prediction is stored as a git-committed artefact, so its existence and timing are part of the version history. A finding with no independent prediction is treated as in-sample-only and tiered accordingly.
• Cross-domain convergence. Findings that lean on a single data source, methodology, or time window are flagged at the validator stage as single-source. The flag is a downgrade signal feeding the verdict decision; it is recorded, not silently dropped.
• Effect-size disclosure. Statistical significance without effect size is treated as a wound, not a finding. The output stage flags a numeric claim that ships without an accompanying effect-size estimate — a warn-level finding surfaced to the reviewer, not a hard render-time block.
• Hypothesis pre-registration. The hypothesis under test is recorded as a structured field before the data work; mid-stream pivots are visible against that record. Acting on a pivot — downgrading the verdict tier — is operator practice reading that record, not an automatic transition.
• Adversarial review. On contested queries, a load-bearing claim is routed to a debate pair: one agent argues the strongest case against it, a mediator weighs both. Debate fires where the query is contested rather than on every claim; a claim that cannot survive deliberate attack is killed.
• Replication availability. Findings that cannot be reproduced from the published artefacts are flagged as ephemeral. The platform tracks which signals have been re-validated and when — this re-validation tracking is code-enforced.
• Falsification path. Every active finding is expected to ship with the conditions under which it would be retired. This is a documentation convention the writer chain follows, not a gate that refuses publication on a missing kill criterion.
• Limitations propagation. A paper that cites a signal automatically inherits that signal’s known limitations into its own caveats section. The audit gate enforces this at write time — a hard, code-enforced check.

Two of these — limitations propagation and re-validation tracking — are gates in the strict sense: code that blocks or records without operator action. The rest are protocols the operator runs, supported by recorded fields and warn-level findings. Presenting all eight as automatic checks would overstate the machine and understate the operator; both do real work here.

§ III   The audit gate

Each publishing arm runs its drafts through an audit gate before publication — the Institute, the newswire, and the warm-agent fleet each have their own. They share a design but are separate code paths, calibrated to their register.

The Institute’s audit gate, the most fully developed of the three, runs a fixed sequence of checks:

• Verdict-tier coherence — a CONFIRMED_WEAK signal cannot be cited as CONFIRMED.
• Citation-strength sufficiency — every load-bearing number traceable to an extant source.
• Figure consistency — the chart matches the text.
• Persona-voice integrity — the byline matches the register.
• Limitations-propagation — caveats from cited signals carried forward.
• Killed-signal cross-check — no claim leans on a signal already retired.

Failure on any check returns the draft to its writer agent with a structured complaint. Repeated failure on the same check, across multiple drafts, raises an audit-pipeline ticket that an operator reviews. The pattern, not the individual draft, is what gets attention.

§ IV   The killed-signals ledger

The Institute publishes a public killed-signals ledger. Every entry on that ledger is an artefact of this pipeline. Operationally:

• A signal that fails its forward-out-of-sample test is moved to SUSPENDED while the analyst chain investigates.
• A signal that fails twice, or fails on a structural ground (effect size collapses, source data is withdrawn, the underlying mechanism is contradicted), is moved to NULL or NOISE.
• The transition is logged with the test that caused it, the analyst chain that observed it, and the supervisor that approved the move. The ledger entry is a derived view of that log.

By policy, a retired signal is not revived without re-running the falsification sequence. A verdict is a field, and revivals have happened — what makes the discipline real is the procedural commitment that a revival must earn its way back through the battery, logged the same way the retirement was. The constraint is editorial, not a technical lock; presented as a technical lock it would be a claim the code does not back. What constrains the operator is not the code but the log: a revival without a re-run of the battery is visible in the ledger, and a ledger that contained one would be the kill rate’s own contradiction.

§ V   Why this matters externally

For anyone evaluating this platform: the durable asset is not the signals. Signals come and go. It is the standard — and the standard is held by an operator who runs these protocols consistently, supported by the gates that are code.

The methodology page on consiliences.com tells you what the rules are. This page tells you which of them are enforced in code and which are operator practice, and that the same standard runs across the publishing arms.

What is being demonstrated here is a working pair: an engine that embeds real falsification gates, and an operator who applies the rest as discipline. Take either away unprepared and the kill rate stops being trustworthy; the mitigations against an unprepared loss of the operator are set out in the FAQ. The point is that the pair is what is on offer — not a content business, and not an autonomous platform, but an operator-plus-engine that produces a falsification-positive output stream at a cost a fifty-researcher institution cannot match.

Drafted with AI assistance under operator supervision; substantive claims are operator-authored or operator-approved.