AI Test AutomationToolsHarness QA tool
AI Test Automation · Tools Preview
Harness QA tool
The harnessqa checks, running in the page on a sample fleet of five client harnesses. Change a channel, an install, a deny rule or a version and see who still gets the release; run a hook the way Claude Code runs it; decide a call against rules from every scope; gate a recorded session. Nothing is sent anywhere.
| Client | Channel | Installed | Auto-update | Project deny rules | Own skill |
|---|
Exit 2 blocks and shows stderr as the reason. Exit 0 lets the call through, with an optional JSON decision on stdout. Any other exit code is a non-blocking error: Claude Code carries on. The hook's logic runs here in JavaScript; the Python file in the fleet is what a harness ships, and a test checks the two agree.
Rules from every scope merge into one list. Deny is checked first, then ask, then allow; the first match in that order decides, whatever its scope and however narrow an allow rule is. A trailing space and star (or :*) matches by prefix: Bash(git diff *) covers git diff main but not git diff-index.
The same checks from a terminal or a pipeline, on your own fleet: write its inventory (fleet.yaml), point it at each client's .claude folder and managed settings, and run:
python -m harnessqa fanout fleet/fleet.yaml # who gets each release; exit 1 if anyone does not python -m harnessqa hooks fleet/fleet.yaml # every hooks config: events, matchers, scripts python -m harnessqa profile fleet/fleet.yaml # usage by client and role, and the test plan it implies claude -p "..." --bare --plugin-dir plugins/core-harness --output-format stream-json --verbose > run.jsonl python -m harnessqa check run.jsonl --plugin core-harness --never "Bash: git push" --no-denials pytest -q tests # hook contracts, settings rules, fan-out, trajectories
The tool, its tests and the sample fleet are in data/testlab/harness. Why each check exists, with the runs behind it: Harness QA.