Write the sampling and perturbation plan

Before any run, write down which inputs are sampled, which perturbations are applied, how the sample is split into groups, how many draws are taken, and what result would count as a failure. The plan is yours to justify; the public record gives no numbers.

The requirement in plain English

Before any run, write down which inputs are sampled, which perturbations are applied, how the sample is split into groups, how many draws are taken, and what result would count as a failure. The plan is yours to justify; the public record gives no numbers.

What the public record says

The public text says sampling widens what a team can see of the input space and may be folded into an existing assessment. It states no sample size, no grouping rule and no threshold.

Why it matters to you

This is the deliverable the cold buyer said it would pay for, because it is the one thing the standard will not hand over. A plan turns a vague intention into an auditable method: it fixes the frame, the generator, the number of draws and the decision rule before the result is known. That order matters. A plan written after the result is a rationalisation, and a reviewer can tell by the dates.

Evidence a reviewer asks for

• The plan, signed and dated before the first run.
• The sampling frame with strata and the reason for each.
• The perturbation catalogue with generators, versions and seeds.
• The licence, provenance and retention note for the test data, with personal data cleared.
• The compute and cost estimate for the campaign, with a budget line.
• The decision rule and the acceptance threshold, with the sign-off.

Checks to run this month

1. Was the plan signed before the first run?
2. Is the generator checked against a model with known behaviour before the full run?
3. Does every stratum have a reason?
4. Is the decision rule independent of the result?

Dates

Start now. Evidence freeze 2027-01-17, thirty days before the planning date.

Cross-references

• High-dimensional input and why exhaustive testing fails
• Set the acceptance threshold and the decision rule
• Corruption robustness and its efficient assessment

What this page does not state

The public text states no sample size, no strata and no threshold.