Quality Authority

Manual testing specialists who serve as the final arbiter of requirements, because the only TRUE way to test is by hand!

Verifying quality is a specialist field that no Code Engineer is qualified to perform. This is not a reflection on the Code Engineer’s intelligence or dedication; it is a recognition that the skills required to build a system and the skills required to verify that system are fundamentally different disciplines. A Code Engineer who tests their own work is like a student grading their own exam: they will inevitably overlook the gaps in their understanding because those same gaps blind them to the deficiencies in their output. Additionally, performing testing impedes the ability of the Code Engineer to do their only job, which is typing code. SADMF addresses this by establishing the Quality Authority as a dedicated team of testing specialists whose sole purpose is to validate that the software meets requirements through comprehensive manual test execution.

Requirements Interpretation

The Quality Authority is the final arbiter of what the requirements mean:

  • Ambiguous requirements — when a requirement is ambiguous, as requirements inevitably are, the Quality Authority interprets it.
  • Conflicting requirements — when a requirement conflicts with another requirement, the Quality Authority resolves the conflict.
  • Implementation disputes — when a Code Engineer implements a requirement differently than the Quality Authority expected, the Quality Authority’s interpretation prevails, because the Quality Authority has studied the requirements more deeply than any Code Engineer, who was focused on typing the code rather than understanding the broader business context.

The Quality Authority maintains a Requirements Interpretation Log that records every interpretive decision, creating an authoritative reference that prevents the same ambiguity from being re-debated in future Convoys.

Manual Test Execution

The Quality Authority creates, maintains, and manually executes test scripts based on their understanding of the requirements. Each test script specifies the exact steps to perform, the exact data to enter, and the exact results to observe. Test scripts are executed by hand because the end-user uses the system manually, and therefore the only TRUE way to test it is manually. Automated tests verify that code runs; manual tests verify that the software works as a human would experience it. The Quality Authority executes each test script exactly as written, recording pass or fail for each step, capturing screenshots of every result, and documenting any deviations in a Test Execution Report. The Test Execution Report is reviewed by the Development Integrity Assurance Team (DIAT) to validate that the testing was performed correctly.

Testing Cycle

The Quality Authority’s test execution cycle is integrated into the Convoy timeline:

  1. The Source Management Team merges all feature branches into the Conflict branch and resolves all conflicts.
  2. The Quality Authority receives the integrated build for testing.
  3. The testing window is defined by the Commodore and typically spans two to three weeks, during which the Quality Authority executes every test script in the regression suite plus all new scripts written for the current Convoy’s features.
  4. Defects discovered during testing are logged in the Defect Tracking Spreadsheet (a dedicated tab in the Release Tracking spreadsheet) and assigned back to the Code Engineers who wrote the offending code.
  5. Code Engineers fix the defects, the Source Management Team re-merges, and the Quality Authority re-executes the affected test scripts.
  6. This cycle repeats until the Quality Authority signs off that all tests pass.

Authority and Metrics

The Quality Authority’s sign-off is a prerequisite for the Change Rejection or Acceptance Party (CRAP) to review the Convoy’s changes. Without QA sign-off, the CRAP will not convene, and the Convoy cannot proceed to Deploy the Fleet. This gives the Quality Authority effective veto power over any release, a power that is appropriate given their role as the organization’s last line of defense against defective software reaching production.

Quality Authority members are measured by:

  • Defects found during testing — more is better, as it indicates thorough testing
  • Production defects found after release — fewer is better, as it indicates effective testing
  • DevOps Process Excellence Assessment scores — adherence to framework practices

The Quality Authority does not ship fast; the Quality Authority ships right.

See Also