The Crucible

圆桌萃鉴

The Crucible is a Codex review protocol that seals separate first passes, compares every finding by stable ID, and adjudicates disagreements against source evidence.

English · 简体中文

Codex skill CI Release License: MIT

One reviewer can miss a problem. Several reviewers can inherit the same assumption and agree too quickly. The Crucible makes the review design visible, preserves each seat's initial judgment, and keeps supported objections alive until the evidence resolves them—or cannot.

It is designed for consequential documents, plans, records, and changes. It does not turn a majority vote into truth or treat review as permission to publish, approve, import, or write.

Quickstart

1. Install

Install the reviewed v0.1.0 release in your Codex skills directory:

git clone --branch v0.1.0 --depth 1 \
  https://github.com/SEKKAIE/the-crucible.git \
  ~/.codex/skills/the-crucible

Python 3.10 or later is required only for the included structured-review comparator.

2. Invoke

Open a new Codex task and write:

Use $the-crucible to review <artifact or decision>.

3. Confirm the configuration

The expected first response is a configuration proposal, not a completed review. It identifies the frozen scope, coverage, evidence, mutation boundary, and each non-duplicative seat:

Seat Role Independent responsibility Required output
1 Lead reviewer Freeze scope, complete a first pass, and adjudicate Decisions and unresolved items
2 Evidence reviewer Check provenance, completeness, and contradictions Evidence gaps and verification requests
3 Domain reviewer Check meaning, edge cases, and consequences Domain findings and proposed corrections

Confirm or revise the table. Substantive review begins only after confirmation.

How it works

  1. Configure the table. Three seats are the default full review, not a fixed panel. Two seats are a labeled lower-assurance peer review; specialized risk may justify four or five.
  2. Freeze the contract. Record stable item IDs, source locations, versions or hashes, decision criteria, and what may not be changed.
  3. Seal first passes. Each seat reviews the same frozen set before seeing any sibling output.
  4. Compare mechanically. compare_reviews.py validates coverage and joins structured findings by item ID.
  5. Adjudicate with evidence. Supported objections outrank vote count. Missing evidence becomes an explicit unresolved item.
  6. Separate review from action. External writes require separate, explicit authorization.

What "independent" means

The Crucible provides process independence: separate seats receive the frozen artifact and review contract, and their first-pass outputs are persisted before cross-review. It does not guarantee statistical or cognitive independence. Reviewers using the same model or provider can share blind spots and correlated errors.

For higher-assurance work, combine distinct models or providers where available, add a qualified human reviewer, and verify material claims against primary evidence. Repeating the same model does not by itself create independent evidence.

Worked example

examples/worked-review contains a synthetic, runnable review packet: source material, a manifest with source mapping, three sealed seat outputs, the generated discrepancy matrix, a consensus artifact, and an unresolved decision.

Run the comparison from the repository root:

python3 scripts/compare_reviews.py \
  examples/worked-review/reviews/seat-1-lead.json \
  examples/worked-review/reviews/seat-2-evidence.json \
  examples/worked-review/reviews/seat-3-domain.json \
  --manifest examples/worked-review/manifest.json \
  --output examples/worked-review/expected/discrepancy-matrix.json

Expected summary: three reviewers cover all three items; one item is a unanimous pass, one is a disagreement, and one remains a unanimous verification request. Two items require adjudication.

Scope and limitations

  • This release is packaged and verified for Codex. Other agent environments are not yet claimed as tested integrations.
  • The protocol structures scrutiny; it cannot authenticate a source, eliminate model bias, or replace qualified legal, medical, security, or other professional judgment.
  • The comparator checks schema, coverage, verdict differences, and evidence flags. It does not decide whether evidence is true or sufficient.
  • User-supplied manifest and evidence locators are preserved for traceability. Sanitize private paths or identifiers before sharing an output.
  • A completed review may still end with unresolved items. That is a valid result, not a failure to manufacture consensus.

Repository map

License

MIT