The Experiment Preflight: 12 Checks Before Work Begins

Gate zero: name the choice this work will inform

Do not begin with a deliverable. Begin with a sentence that names the choice due after the experiment: fund the next stage, change the audience, replace a mechanism, gather a different class of evidence, or close the idea. Add the decision date and the person authorized to make it. If no plausible result would alter that choice, the proposed work is validation theater rather than an experiment. The Scrum Guide links inspection with adaptation, which is why a review meeting without decision authority fails this first gate.

This preflight is a go, revise, or stop control, not a scoring game. Every required item needs dated evidence or an explicit 'unknown.' Unknowns may be acceptable when exposure is low and the test is designed to address them; they are not silent passes. Complete the checks before booking participants, purchasing software, publishing promises, or building automation. The checklist improves traceability, but it cannot supply specialist judgment for regulated, safety-critical, health, employment, or privacy-sensitive work.

Evidence: Scrum Guides; National Academies of Sciences, Engineering, and Medicine

Checks 1–3: isolate the assumption and evidence rule

Check 1: write one load-bearing assumption, not a bundle such as 'people want it and can use it and will pay.' Check 2: describe an observable trace expected if the assumption is credible, plus a trace that would weaken it. Check 3: state the interpretation rule before collecting data. The rule may be qualitative—for example, the same unprompted breakdown appearing across contrasting cases—but it must be specific enough that another reviewer can recognize whether it occurred.

Verify that the proposed observation bears on the assumption rather than an easier neighboring question. General interest does not establish repeated use; task completion does not establish value; a smooth facilitator-led session does not establish independent comprehension. GOV.UK's alpha guidance is useful here because it foregrounds risky assumptions, prototypes, and measures of success. Record why the chosen evidence is proportionate and list at least one plausible alternative explanation for the same result.

Evidence: Scrum Guides; UK Government Digital Service

Checks 4–6: cap exposure and preserve reversibility

Check 4: set separate ceilings for calendar days, staff hours, spending, and number of people exposed. Check 5: define the rollback route, including removal of test material, correction of misleading information, and support for anyone affected. Check 6: list dependencies whose delay could leave the test running by default, such as a platform review, contractor handoff, data export, or manager approval. Assign a stop authority who does not need to seek new permission when a ceiling is reached.

Use an explicit workflow to reveal work that would otherwise hide between states. The Kanban Guide supports making workflow, work in progress, and flow measures visible, which can expose an 'experimental' service accumulating unresolved cases. However, an efficient queue is not proof of a useful idea. Inspect burden from the participant's perspective as well: time, accessibility, disclosure, emotional effort, privacy, and difficulty exiting. A cheap internal test can still be expensive for the people asked to absorb its uncertainty.

• 4. Time, labor, cash, and audience ceilings are listed separately.
• 5. Rollback, correction, deletion, and participant support paths are workable.
• 6. Every external dependency has an owner, timeout, and safe default state.

Evidence: UK Government Digital Service; Kanban Guides

Checks 7–9: protect participants and data quality

Check 7: confirm that people know what participation involves and can decline or leave without penalty. Check 8: collect only information needed for the stated observation, define access and deletion, and avoid sensitive data when a less intrusive trace works. Check 9: document recruitment criteria, facilitation prompts, excluded cases, missing observations, and any mid-test change. These fields make it harder to treat a convenient sample or heavily coached performance as representative evidence.

The National Academies' synthesis on learning highlights the importance of prior knowledge and context. Use that insight operationally by asking who the test omits and whether participants possess experience that the intended audience may not. Do not 'correct' contradictory cases out of the record. Store observations separately from interpretation, minimize personal details, and follow applicable consent and data rules. If those obligations are uncertain, pause and obtain appropriate guidance rather than using this editorial checklist as authorization.

Evidence: Scrum Guides; Kanban Guides; National Academies of Sciences, Engineering, and Medicine

Checks 10–12: lock the review and closure protocol

Check 10: schedule the decision meeting and require the evidence owner, decision owner, and someone able to challenge the interpretation. Check 11: prewrite the available dispositions—advance, revise the mechanism, seek a different observation, or close—and the authority required for each. Check 12: create an automatic closure step that archives the evidence, records the decision, ends access, and assigns any correction. A calendar end without these actions merely stops collection; it does not close the experiment.

Define red conditions that end work immediately rather than waiting for review: breached exposure caps, invalid consent, unsafe behavior, unverifiable records, or an inability to reverse a consequential change. Define amber conditions that trigger redesign, such as a shifted audience, altered prompts, missing comparison cases, or a threshold that no longer matches the decision. Green means the protocol is ready to run, not that the hypothesis is true. No aggregate score should override a red condition.

• 10. Review attendees, evidence package, and decision time are already booked.
• 11. Advance, redesign, gather-different-evidence, and close outcomes are all legitimate.
• 12. Archival, access removal, correction, and follow-up have named owners.

Evidence: UK Government Digital Service; National Academies of Sciences, Engineering, and Medicine

Complete the preflight record and set its expiry

Place the twelve answers in a dated preflight record with source links, version identifiers, unresolved questions, and initials from the responsible owners. Give the record an expiry tied to the fastest-changing dependency. A plan based on current platform behavior may expire sooner than one based on a stable internal process. If the start date slips beyond expiry, recheck the affected fields instead of copying the old approval. Keep the superseded record so later reviewers can reconstruct which assumptions were reasonable at the time.

For immediate use, take a proposed experiment and answer Gates 0 through 3 first. If you cannot name the decision or precommit an evidence rule, stop there; more logistical detail will not repair the design. If those gates pass, complete exposure, participant, and closure checks with the people who own the relevant risks. This artifact records preparedness for one bounded inquiry. It does not certify future performance, durable demand, legal compliance, or the absence of harm.

Evidence: Scrum Guides; National Academies of Sciences, Engineering, and Medicine