Regression test cases for a release are selected from the product change, not from memory. A release regression run is evidence for one change: it must execute the test cases that were brought in line with that change and approved, not a selection someone assembles the day before the release. When the run and the change are not connected, a green run says nothing about what actually shipped.

Why a regression run picked by hand proves little

  • Stale cases pass. A case written before the change still describes the old behaviour. It can pass against the new product and confirm nothing, or fail and send someone to debug a test that was simply out of date.
  • Affected cases hide. A change to how invitations work touches cases whose titles never mention invitations. Selecting regression test cases by title or by memory misses them; finding them takes a search by meaning, the one semantic impact analysis describes.
  • Nobody can trace it. A week later, “what did we test for this change?” has no answer, because the selection lived in someone’s head.

One changeset, reviewed once

maintain-test-cases is the skill in the QA Vault harness that brings test cases in line after a product change: an open-source plugin for Claude Code and Codex CLI you install in one command. It works as one change-driven sync:

  1. Find everything the change touched. It searches the vault by meaning and by title, and reads each candidate in full before deciding it is affected.
  2. Stage one changeset. Updates to existing cases, new cases for new behaviour, and removals of cases whose feature is gone, together in one draft. Nothing is written to QA Vault yet.
  3. Flag what looks wrong. Behaviour in the change that a QA Engineer would pause at is marked for validation instead of quietly becoming an expected result.
  4. A QA Engineer approves. They edit the draft, reject what is out of scope, and approve the rest. Only then is the changeset applied.

The review happens once, on the whole change, instead of case by case in a hurry before the release.

From the approved changeset to the run

The approved selection is saved as a run template: a named set of links to the cases, not copies of them. A run seeded from the template always receives the current, approved state of every case, and once created it stands on its own, so editing the template later never rewrites a run.

  • The release regression run is created from that template. Every recorded result keeps an immutable snapshot of the case as it was when it was executed, so a later edit to the case cannot change what the run says was tested.
  • The automated part runs from the same template. run-automated-tests executes the Playwright specs linked to those cases and records them as their own automated run, with flake reported separately and real failures handed to healing against the case.
  • The numbers for both come from the same place: what passed, what failed and what is still waiting for a person.

Selection, approval and execution all point at the same approved cases, so the release evidence traces back to the change that caused it.

What the QA Engineer gets

One changeset to review for the whole change, with the doubtful behaviour already flagged. One template that turns the approval into the run, with nothing to reassemble. And runs that still answer “what did we test for this change?” long after the release, because every result carries the case it was tested against.

Where to start

  1. Start from the change, not the run. When a feature changes, bring its cases in line first; selecting the regression test cases comes after, and it is the approved changeset.
  2. Review the changeset as a whole. Updates, additions and removals together, approved once.
  3. Save the approved selection as a template and seed every run for that release from it, manual and automated.
  4. Install the QA Vault harness, and maintain-test-cases and run-automated-tests carry this flow for you.