QA Skills

QA Skills is a portable, deterministic quality-assurance runtime for Codex, Claude Code, and Cursor. It combines versioned artifact contracts, a TypeScript CLI, a Playwright Runtime Browser Driver, evidence and defect operations, release-gate reporting, and one canonical cross-agent Skill Bundle.

The runtime never calls an LLM. Agents may author requirement and testcase drafts, while QA Skills validates, registers, executes, and reports immutable canonical artifacts.

• Requirements
• Install
• Quickstart
• Deterministic demo
• Using it from an agent
• CLI reference
• Configuration
• Two execution lanes
• Consuming the gate in CI
• Environment and side-effect safety
• Evidence and redaction
• Artifact layout
• Troubleshooting
• Compatibility
• Development and governance

Requirements

• Node.js >=22 (package.json engines). CI runs 22 and 24; those are the versions the project tests on.
• npm
• A locally installed Chromium binary for browser execution

QA execution never downloads a runtime or browser implicitly.

What does run at install time, and it is not this package. @gwinnguyen/qa-skills declares no preinstall, install, postinstall, or prepare script, so nothing of its own executes when you install it. One dependency does: sharp, used to compose annotated screenshots, runs node install/check.js || npm run build. On a machine with a system libvips, or with npm_config_build_from_source set, that fallback compiles libvips from source. This is stated because the sentence above is otherwise easy to read as covering the whole install, and it does not.

Install

npm install --save-dev @gwinnguyen/qa-skills
npx playwright install chromium

The package installs a qa-skill bin. Call it through the local path so a QA gate can never run on a binary nobody pinned:

./node_modules/.bin/qa-skill runtime verify

{"executable":"/path/to/project/node_modules/.bin/qa-skill","version":"1.0.3","range":">=1.0.0 <2.0.0","compatible":true}

Do not invoke QA Skills through a bare npx qa-skill. With no local install present, npx falls back to fetching from the registry, which silently swaps the binary your evidence was produced by. Every example below uses ./node_modules/.bin/qa-skill; shorten it with a shell alias or an npm script if you like, but keep it resolving locally.

git clone https://github.com/dangnhit/qa-tester.git
cd qa-tester
npm ci
npm run build
npx playwright install chromium
node dist/src/cli/index.js runtime verify --range ">=1.0.0 <2.0.0"

Quickstart

Every command below was run end to end against a fresh project; the outputs are the real ones.

1. Initialize

./node_modules/.bin/qa-skill init

Success is silent. It creates qa.config.yaml if absent and ensures qa-results/ is ignored:

# qa.config.yaml
version: 1

See Configuration for headers and Test Data hooks.

2. Describe the environment under test

Every run is bound to an Environment Profile. The minimal valid one:

{
  "artifactType": "environment-profile",
  "schemaVersion": "1.0.0",
  "producerVersion": "1.0.3",
  "environmentProfileId": "ENV-LOCAL-1",
  "name": "Local development",
  "classification": "local",
  "baseUrl": "http://127.0.0.1:3000",
  "productionReadOnly": false
}

Save it as environment.json. classification is one of local, test, staging, production, and it decides what side effects the run may perform — see Environment and side-effect safety.

3. Generate the four agent-authored drafts

You never write these from scratch. draft init prints a valid skeleton for each:

Q=./node_modules/.bin/qa-skill
$Q draft init --type requirement-analysis > requirement.json
$Q draft init --type test-plan           > plan.json
$Q draft init --type test-case           > testcase.json
$Q draft init --type coverage-obligation > coverage.json

Replace the PLACEHOLDER: strings with your own content. Two fields need a word of explanation:

• requirement-analysis authority is re-derived from the text, not taken on trust. A statement declared AUTHORITATIVE must contain a modal — must, shall, should, required, expected, need(s) to — or the ingest refuses with Requirement authority AUTHORITATIVE disagrees with provenance-derived ASSUMED, exit 3. The full derivation table is in artifact-authoring.md.
• test-case revisionId / instanceId are author-supplied identity anchors. Any stable string works; qa-skill fingerprint --file testcase.json prints a content digest that is a good choice.

coverage-obligation's requirementAnalysisArtifactId placeholder is fine as-is — workflow bootstrap rewrites it to the artifact it just registered.

4. Register the planning bundle

$Q workflow bootstrap --root . \
  --environment-file environment.json \
  --requirement-file requirement.json \
  --plan-file plan.json \
  --test-case-file testcase.json \
  --coverage-file coverage.json

{"runId":"20260809T052958Z-2390ef","bundle":{"sourceRunId":"20260809T052958Z-2390ef","artifacts":[{"artifactId":"01KZJFZEDJDEYPT797NA2FJ927","sha256":"560b9fa0..."}]}}

This is one atomic, checksum-bound planning run. Keep its runId — it is the source every later run cites.

5. Scaffold an input and run

PLAN_RUN=20260809T052958Z-2390ef   # the runId from step 4

$Q workflow scaffold --root . --mode plan \
  --source-root . --source-run-id "$PLAN_RUN" \
  --environment-file environment.json \
  --output plan-input.json

$Q workflow run --input plan-input.json

{"runId":"20260809T052959Z-6cc159","mode":"plan","outcome":"COMPLETED","operationOrder":["ingest-requirement-analysis","ingest-testcases","ingest-coverage-obligation"],"validation":{"valid":true,"diagnostics":[]}}

plan mode launches no browser. Move to full once you have a real baseUrl to drive.

6. Re-verify any run, at any time

$Q validate --root . --run-id 20260809T052959Z-6cc159

{"valid":true,"diagnostics":[]}

Deterministic demo

The demo needs the checkout, not the npm package — it ships its own fixture server.

git clone https://github.com/dangnhit/qa-tester.git
cd qa-tester
npm ci
npx playwright install chromium
npm run demo

The demo binds an ephemeral 127.0.0.1 port and makes no external request. Its fixture deliberately leaves an authoritative validation message empty, emits QA_DEMO_CONSOLE_ERROR, and calls a local endpoint that deterministically fails. The runtime executes Chromium desktop and emulated mobile Test Case Instances, records traces, sanitized raw and annotated screenshots, console/network telemetry, product Bug Candidates, a QA report, and a validated Full Artifact Profile. It also creates one owned synthetic Test Resource and proves its lifecycle through a separate linked Cleanup Run.

The command exits 0 only when the intentional defect is detected as FAILED + PRODUCT_DEFECT, the QA Run is COMPLETED_WITH_FAILURES, the release recommendation is NOT_READY, each desktop/mobile attempt has its required evidence, cleanup completes, and all expected artifacts validate. Canonical run data is written under qa-results/; convenient copied evidence projections are written under demo-artifacts/. Both are ignored.

Using it from an agent

The CLI above is the whole runtime, but the intended entry point is an agent driving it through the Skill Bundle.

Install the bundle

Project installations are recommended because runtime binding and review travel with the repository:

./node_modules/.bin/qa-skill skills install --agent claude --target project
./node_modules/.bin/qa-skill skills install --agent codex  --target project
./node_modules/.bin/qa-skill skills install --agent cursor --target project

The roots are .claude/skills, .codex/skills, and .cursor/skills. Use --target user for the corresponding directory under the user home. The installation manifest binds the runtime command, real path, resolution source, version, and executable checksum; skills verify fails closed with a typed Runtime Binding status if any of that identity is missing, changed, or incompatible. After source updates, run skills verify, then skills update; never patch an installed copy directly.

Then ask for it by name

Use the qa-tester skill in full mode against the local test environment.
Treat the acceptance criteria in docs/profile.md as authoritative.
Do not perform external or destructive side effects.

qa-tester orchestrates the Full QA Lifecycle. Ask for it when requirements, test design, controlled data, browser execution, evidence, defects, and reporting should stay in one immutable QA Run.

Standalone adapters call the same typed QA Operations, and can be asked for individually:

Use the evidence-collector skill for run <run-id> and attempt <attempt-id>.
Capture only the channels permitted by the registered evidence policy.

Every path an install writes

Two of the three agents need a discovery shim outside the skills root, because neither reads a bare skills/ directory (ADR-0011). Both are recorded in the manifest and checksummed like any other installed file, and skills uninstall reverses both.

AGENTS.md is a file you own, and --agent codex edits it in place. It is the one path here that touches pre-existing content, so it is bounded on every side: the install owns only the text between its <!-- qa-skills:start … --> and <!-- qa-skills:end --> markers, never touches a byte outside them, and refuses outright — before any file is written — when the markers are already malformed, rather than guessing which pair to overwrite. skills verify checksums only the managed region, so your own edits around the block are not drift. skills uninstall strips the block and leaves the rest of the file as you wrote it, deleting AGENTS.md only if the block was the entire file. --agent cursor creates a file it owns outright; that one is deleted whole on uninstall.

Codex reads global instructions from its Codex home (~/.codex/AGENTS.md, or $CODEX_HOME), which is why the user-scope shim goes there rather than to ~/AGENTS.md.

Removing it

./node_modules/.bin/qa-skill skills uninstall --agent claude --target project
npm uninstall @gwinnguyen/qa-skills

skills uninstall removes owned unchanged files and reports drift leftovers rather than deleting a file you edited. qa.config.yaml, qa-results/, and the qa-results/ line init appended to .gitignore are yours to remove by hand — nothing deletes recorded run evidence for you.

CLI reference

Commands that produce output use machine-readable JSON unless noted. Successful qa-skill init and qa-skill artifact ingest are intentionally silent on stdout.

Public workflow modes are plan, execute, full, exploratory, retest, and regression. cleanup is a linked maintenance-run profile created through the cleanup operation; it is not accepted by the public workflow runner.

Exit codes:

Configuration

qa.config.yaml is discovered by walking upward from the working directory to the repository root, and the nearest file wins outright. The minimal file is version: 1; two optional blocks exist, headers and hooks.

Project configuration and Test Data Hooks must be reviewed source files — an executable config is refused by design. Store only Secret References (${ENV:VARIABLE}) in inputs; resolve secret values in memory at execution time.

→ Full configuration reference

Two execution lanes

A test result satisfies a Coverage Obligation only when the QA Runtime observed the run that produced it.

• Lane 1 — the runtime drives the browser over a bounded Test DSL and registers one test-result per attempt.
• Lane 2 — qa-skill execute playwright starts your own committed Playwright suite, captures its exit status and JSON report, and registers one test-result-batch anchored to the commit and a checksum of the committed spec tree.

A result file handed to the runtime by any other route stays an Agent Draft: reportable, never coverage-crediting.

Lane 2 identifies a spec by a tag in its test(...) title — [qa:<testCaseId>/<revisionId>/<instanceId>@<surface>]. Lane 2 is an anchoring and provenance mechanism, not a sandbox: it binds a committed, human-merged spec tree to an execution this runtime started and whose exit it saw, and certifies nothing about the code that authored the report.

→ Every refusal, the pause/resume flow, and what lane 2 does not prove

Consuming the gate in CI

qa-skill export projects a finalized run's persisted release gate into JUnit XML or SARIF 2.1.0 and writes a provenance sidecar beside it. It exits 0 on success including when recommendation is NOT_READY — the verdict travels in the projection, never in that exit code.

Two pitfalls silently turn a failing gate green in GitHub Actions: workflow run exits 1 on NOT_READY, which makes every later step without if: always() skip; and | tee swallows that exit code unless the step sets set -o pipefail. Both are worked through, with a copy-pasteable pipeline, in the full reference.

→ Consuming the gate in CI

Environment and side-effect safety

• production requires explicit read-only opt-in and permits only none side effects.
• reversible operations require owned Test Resources and an idempotent cleanup action.
• external operations require a scoped, expiring External Effect Permit.
• Destructive actions, real payments, wildcard recipients, arbitrary shell hooks, and undeclared environments are denied.
• Every browser attempt gets a fresh context. Emulated mobile is not a real-device or cross-browser claim.
• Test Data Hooks are pre-registered typed capabilities; agents do not improvise setup or cleanup commands.

The full reference is safety.md.

Evidence and redaction

Evidence listeners start before the actions they observe. Protected targets persist Sanitized Raw Evidence only; annotations are derived separately. Mandatory selectors or regions are masked before screenshot bytes are registered. After any secret is resolved, screenshots require provable secret-derived masking regardless of environment classification; otherwise the runtime registers an Evidence Gap without creating PNG bytes. Other unsafe captures likewise become Evidence Gaps.

Trace retention is opt-in

A browser trace archive embeds un-provably-redacted DOM and network content, so trace retention is off by default and no trace is even started unless the environment permits it. To opt in, set evidenceProtection.retainTrace: true on the Environment Profile and raise the trace evidence mode above off (for example on-failure, always, or required). Even when retention is permitted, a trace is still refused — recorded as an Evidence Gap instead of an archive — if a secret was resolved during the attempt or the environment is protected. Declaring any domSelectors or regions redaction target makes the environment protected (no archive channel can prove that target was masked), so traces are never retained there. When retainTrace is absent or false, nothing is captured and no gap is recorded — the environment has simply opted out.

Known secrets are scrubbed from errors and telemetry. Never put resolved credentials, cookies, personal data, or production payloads in testcases, examples, bug reports, or logs. The checked-in examples/ use fixed synthetic identifiers and .test-style data.

Artifact layout

The manifest, not the directory layout, is authoritative:

qa-results/<run-id>/
├── run-metadata.json
├── artifact-manifest.json
├── inputs/
└── evidence/

demo-artifacts/<run-id>/
├── screenshots/raw/
├── screenshots/annotated/
└── traces/

Every canonical descriptor and binary is registered in the manifest with a checksum; relationships use artifact IDs. demo-artifacts/ contains convenience copies only and is never authoritative. Consumers must not scan for the newest run or guess filenames. A completed run is immutable; retest, regression, and cleanup create linked runs.

Troubleshooting

• qa-skill: command not found: the bin is not on PATH after a local install. Call ./node_modules/.bin/qa-skill, not a bare qa-skill.
• Requirement authority AUTHORITATIVE disagrees with provenance-derived ASSUMED: the statement's normalizedText carries no modal verb, or its sourceProvenance.kind is code (which always derives INFERRED). See the derivation table in artifact-authoring.md.
• Chromium executable missing: run npx playwright install chromium during setup. QA execution itself will not download it.
• Runtime missing/incompatible: install the pinned package locally and run qa-skill runtime verify; do not use a remote fallback.
• AWAITING_RUNTIME: provide the configured browser/test-data service IDs and resume the same nonterminal run.
• AWAITING_HUMAN_INPUT: have the named person run the command pendingHumanInput identifies (qa-skill approval record or qa-skill attestation record) against the paused runId, then resume the same nonterminal run — see recovery.
• AWAITING_OBSERVED_EXECUTION (exit 2): the run is paused, not failed. See Two execution lanes.
• Live lock: confirm no active process owns the run; do not delete lock files blindly.
• Artifact validation failure: inspect normalized diagnostics, fix the canonical JSON or source draft, and generate a new artifact. Do not edit Markdown projections.
• Evidence Gap: repair the capture/redaction policy or selector and create a new attempt; never substitute an unregistered file.
• Installer drift: review skills verify; use skills update --force only after intentionally accepting local replacement.
• Demo returns nonzero: confirm Chromium is installed and that local loopback connections are permitted, then rerun npm test -- tests/e2e/demo.test.ts.

Compatibility

Starting at v1.0 the public contract is exactly two surfaces: the named exports re-exported from ., and the CLI commands in the reference table above with their documented flags and exit codes.

import { createQaTester } from "@gwinnguyen/qa-skills";

Everything else — dist/ layout, internal module boundaries, helper names, and any artifact JSON shape beyond its published JSON Schema — may change in a minor release.

→ What v1.0 freezes, in full · CHANGELOG

Development and governance

Run the full local gate:

npm ci
npm run generate:types
npm run check:generated
npm run typecheck
npm run lint
npm test
npm run demo
npm run build
npm run smoke:package

• Documentation index
• CHANGELOG
• CONTRIBUTING · CODE_OF_CONDUCT · SECURITY
• Report an issue

Licensed under Apache-2.0; see LICENSE and NOTICE.