# Human and non-human QA

Status: draft. Owner: Andrew Flett. Last reviewed: 2026-09-12.
Canonical: https://agentic.madebymany.com/positions/human-and-non-human-qa

Automated checks confirm nothing has regressed. They do not confirm the thing works, which is a judgement, and judgement is what the human layers are for. We keep both layers open on every project and we do not let one impersonate the other.

## Gates enforce what was agreed

The automated half runs on every change without being asked: formatting, linting, types, tests, accessibility scans, visual regression, a performance budget, a dependency scan. Each is a threshold somebody agreed at the front door of the engagement and recorded in the project's stack manifest: the browsers we support, the accessibility standard, the Core Web Vitals numbers, the licences allowed. A gate answers one question, did this change break the agreed floor, with a pass or a fail. It has no opinion in it and it has to be fast. A commit hook that takes twenty seconds gets bypassed, and a bypassed gate is worse than none because it looks like coverage.

## Audits assess, and a person decides

An audit is a different instrument. It scores how accessible, how fast, how visible or how healthy a thing actually is, names the causes, proposes an order to fix them, and a person reads it. It runs at moments rather than on every change: at the start of an engagement, on whatever we inherit; before a promotion to production; and on a live estate when a client asks how it is doing. An audit that finds problems cold on our own work means the floor upstream failed. On an inherited estate, finding them is the point.

Code quality lives here entirely. Almost everything in it is opinion, and a gate built from opinion slows the work down and gets worked around. Security splits in two. A secrets scan at commit and a dependency vulnerability scan at pull request are not opinions, and they run as gates. Static analysis, the running app and its headers are read by a person, as an audit.

## A finding is a work item, not a test

The instinct after an audit is to write a check so the finding can never come back. We have done this and stopped. A check written from one finding asserts the symptom, and the symptom is not the rule. On this site a metadata check asserted that no two pages shared a description, caught one real defect, and failed the day two pages honestly shared one. A token drift check was worked around and then deleted, and took the guarantee it did give with it. A finding goes to the tracker, where someone weighs it against everything else and decides. The gates stay the baselines and nothing else.

## Milestones are scheduled, not remembered

A milestone that is a sentence in a process document is forgotten until someone reaches for the promote button. So the audits are filed as work items when a production engagement is set up, with their moments attached, and the sign-off before production records which dated reports it relied on. The gate checks that the reports exist. It does not run them.

## What stays human

Whether a journey can be completed, whether a screen reader conveys the page, whether the work is any good: all judgement, all human, and each with a named owner. One QA specialist embedded in the team owns the passes machines are weak at, and a senior engineer reviews every change before it merges, whether a person or an agent raised it. Agents propose; humans merge and promote. Nothing reaches production without a person having decided it should.

## Where we are unsure

We have not measured how often the audits find what the gates missed, so the claim that baselines plus milestone audits cover more than a gate for everything rests on projects we have watched and on the two checks we deleted. The line between a baseline and an opinion also moves. A contrast floor is a baseline today and whether the right colour was used is not, and we expect to move things across that line as we learn.

## Related agentic skills

- [mxa-verify-pr](https://agentic.madebymany.com/playbook/skill/mxa-verify-pr)
- [mxa-verify-promote](https://agentic.madebymany.com/playbook/skill/mxa-verify-promote)
- [mxa-audit](https://agentic.madebymany.com/playbook/skill/mxa-audit)
- [mxa-track](https://agentic.madebymany.com/playbook/skill/mxa-track)