Harness Engineering Audit

Know whether an AI coding agent can work safely in a repository before letting it change anything.

Harness Engineering Audit is a Codex skill backed by standard-library Python. It inventories the instructions, documentation, validation paths, agent tooling, and artifact policies that shape automated engineering work. The result is a scored, evidence-linked assessment with prioritized fixes and a concrete follow-up handoff.

The default is deliberately report-only. It writes audit artifacts, but it does not edit application source, run agents, install tools, or change configuration.

What it answers

Question Evidence inspected
Can an agent find the source of truth? Root and nested AGENTS.md, docs indexes, vocabulary, decision records, and progressive-disclosure guidance.
Can the repository prove a change is safe? Package and Make targets, CI workflows, test/build/lint signals, and automated-versus-human review boundaries.
Are agent integrations controlled? Codex config, skills, MCP declarations, hooks, rules, OMX artifacts, and cross-agent configuration.
Can product intent become verified software? Stories, screen contracts, routes, APIs, design tokens, component contracts, runtime evidence, and visual QA.
Is automation debt visible? Generated artifacts, scaffolding markers, lifecycle policy, task handoffs, recovery guidance, and observability signals.

The audit scores 17 harness dimensions, classifies repository maturity, detects stack-specific work lanes, and separates low-risk documentation fixes from changes that need explicit review.

flowchart LR
    repo[Repository] --> inventory[Static inventory]
    inventory --> score[Score and classify]
    score --> reports[Reports and next step]
    reports -. explicit setup mode .-> setup[Provenance-marked docs and templates]

Quickstart

Requirements: Python 3.10+ and a GitHub CLI release that includes gh skill.

From the repository you want to audit:

gh skill install ryne2010/harness-engineering-audit \
  skills/harness-engineering-audit \
  --agent codex \
  --scope project

python3 .agents/skills/harness-engineering-audit/scripts/run_audit.py \
  . --mode audit

The report set is written to .codex/reports/harness-engineering-audit/. Each run replaces the previous generated report set so stale findings cannot masquerade as current evidence.

The default update check contacts GitHub through gh and records the result without changing the installed skill. Add --no-check-update for an offline audit.

What you get

Artifact Purpose
report.md Overall verdict, dimension scores, strengths, and weakest surfaces.
findings.md and recommended-fixes.md Evidence-backed gaps grouped by confidence, priority, and risk.
scorecard.json and readiness files Machine-readable lifecycle, stack, lane, and docs-to-production assessments.
next-step.md, next-step.json, and prompts/ A resumable planning or execution handoff without a long copy/paste prompt.
upgrade-recommendations.* Approval-gated tooling proposals with source trust and rollback fields.

Safety and limits

Audit mode is a static assessment. It does not execute the target repository's tests, prove runtime behavior, or establish security or production readiness. Scores are planning signals, not certification.

The analyzer reads local repository text and records the absolute repository path, filenames, validation command names, and detected configuration surfaces. Review generated reports before sharing them outside your team. A custom non-empty output directory is not overwritten unless it is already audit-owned or you explicitly pass --force.

Setup modes are opt-in. They can create bounded, provenance-marked harness docs and templates; full-orchestration may also add project agent definitions. Existing non-owned files are treated as collisions, and setup runs emit a rollback manifest. These modes still do not start agents or execute live install/configuration commands.

Tool recommendations come from a bundled catalog. The local script marks them as not web-verified, requires human approval, and never runs the proposed commands.

Development

Run the canonical repository checks and package validation before publishing:

make validate
gh skill publish --dry-run

make validate compiles the Python scripts, checks parity between the source skill and plugin mirror, and runs the skill and release-workflow smoke suites.

See Installation, Usage and modes, plugin distribution, and publishing for operator detail.

Released under the MIT License.