Design a risk-based test strategy across unit, integration, end-to-end, regression and acceptance

Start from risk, not a layer label

Consider a small change to a pricing service. It touches three areas: a pure price function, a persistence adapter, and API-to-service wiring. The change brief names four concerns: cent rounding must not drift, the stored record must preserve its data contract, the API request must actually reach the new logic, and a historical defect must not recur. An approved acceptance rule also defines behavior that a human reviewer may accept.

A common shortcut is to choose a testing pyramid or a fixed count ratio before analyzing those risks. This lesson does not prescribe a ratio. The target artifact is Risk-Based Test Strategy Record v1. For every material risk it records the observable failure, test boundary, oracle authority, fixture, sensor, test purpose, trade-off, and residual risk.

The lesson is planned for about 85 active minutes. The hard prerequisite is the ability to identify relevant seams from a bounded change surface. If that skill is fragile, repair only the part needed here. Source-verification discipline is useful when you choose an oracle authority, but this lesson does not reteach documentation verification.

Bounded promise: you can design an evidence portfolio proportionate to risk and explain what each sensor does and does not establish. The lesson does not claim absolute correctness, one universally optimal distribution, inherent superiority of E2E tests, or that acceptance is synonymous with E2E.

Choose the route that matches your evidence

All routes lead to the same final evidence bar.

• Foundation: use explicit definitions, a complete worked trace, and a partly structured record before independent work.
• Repair: if boundary, oracle, or purpose-versus-boundary reasoning is fragile, repair only that discriminating concept with a counterexample.
• Experienced / Fast Path: skip repetitive explanation when prior evidence is strong and move to independent practice. The artifact, critical dimensions, human decision, and transfer still remain mandatory.

A compact diagnostic: for the opening scenario, decide whether “regression” names a technical execution boundary or a purpose. Then name the authority behind one expected value. Finally identify a risk that a unit-only test cannot see because the relevant seam is outside its boundary. If those distinctions are uncertain, use the Foundation route.

Mental model: the evidence chain

Read a test strategy as this chain:

risk → observable failure → boundary → fixture → oracle → sensor → result → human decision → residual risk

A risk is a consequential failure you do not want to leave unsupported. An observable failure is what you could see or measure if that risk materializes. The test boundary says which components actually participate in execution. A fixture controls state and data so the observation is repeatable. An oracle is the basis for deciding whether the observed behavior is acceptable. The sensor is the test or probe that produces the observation. A sensor result is not yet a decision: a person still has to judge the strength and limits of the evidence.

Unit/local, integration, and end-to-end mostly describe execution boundary and realism. Regression and acceptance mostly describe purpose and authority. A regression test can be local when the historical defect lived at a local seam. An acceptance check does not inherently require UI, network, and full infrastructure; it must exercise enough of the system to support the approved acceptance claim.

Do not confuse an assertion with an oracle. assert total == 64.35 is only a comparison mechanism. The important question is why 64.35 is the authorized expectation. The authority might be an approved business rule, a domain invariant, an adapter contract, or a historical defect record. If an expected value is copied from current program output, a green test can faithfully confirm the same defect.

Foundation: make every risk reviewable

Before writing a test file for a risk, close four questions:

1. What exactly would be observable if this risk happened?
2. What is the cheapest boundary that actually exercises the required seam?
3. What authority makes the expectation defensible?
4. What blind spot remains after this sensor passes?

“Cheapest reliable sensor” does not mean “always unit.” If the risk concerns persistence serialization, a pure-function test is cheap but does not execute the relevant seam. If the risk concerns only a pure calculation invariant, running browser, API, database, and network may add realism that is unrelated to that claim and make localization slower.

Coverage is also only structural execution information. High coverage cannot establish that the oracle was correct, the edge data was discriminating, or the relevant seam was exercised. Treat it as a possible supporting signal, not a substitute for the risk-to-decision chain.

A Risk-Based Test Strategy Record should be explicit enough for a reviewer to ask and answer: Which risk is this sensor for? Why this boundary? Where did the expected behavior come from? What remains unseen?

Deep explanation: a portfolio shaped by seams

The matrix below is an example of reasoning for this scenario, not a reference distribution to copy.

Two axes must remain distinct. “Integration” describes what runs together. “Regression” describes why a sensor is retained to detect recurrence. One test can therefore be both an integration test and a regression sensor. “Acceptance” identifies decision authority and purpose; it does not by itself dictate the technical level.

Instructional asset TST001-A01 represents the same chain without layer percentages. Its text equivalent is: begin with risk; name the observable failure; widen the boundary only enough to exercise the necessary seam; control the fixture; record oracle authority; run the sensor; interpret the result with its limits; let a human make the decision; keep residual risk visible.

Worked examples

Example 1: rounding in a pure function

Suppose the change only affects rounding behavior. The risk is a financial difference at decimal edges. A local sensor with carefully chosen edge inputs directly exercises the relevant seam. A broad test might observe the same behavior, but more components participate and failure localization is harder. When the claim is only the rounding rule, the smaller boundary gives more focused evidence.

Example 2: all-E2E versus a risk-shaped portfolio

One team writes E2E tests for all four risks. Those tests may observe several failures, but feedback is slower, fixtures are more complex, and failures are less localized. Another team writes only unit tests and never exercises persistence or wiring. The desired answer is not a numeric compromise. It is a set of sensors in which every material risk exercises the seam that matters and exposes the remaining blind spot.

Example 3: green test, wrong oracle

A defect changes an expected 64.47 result to 64.46. A developer edits the expected value to 64.46 because “that is what the program returns now.” The test becomes green, but the expectation has lost its authority. The repair is to bind the expected value to an approved rule or invariant and test the implementation against that authority.

Example 4: persistence moves behind a different boundary

In a later version, persistence moves from an in-process adapter to a separate local service. The risk “store the quote correctly” remains, but the seam changes. A local price sensor may remain unchanged, while the persistence integration sensor must move. Transfer means redesigning the affected sensor from the new risk/seam relationship rather than copying the old test file.

Seductive shortcuts and how to reject them

“70/20/10 is always the correct architecture.” A fixed ratio ignores system context, risk, and feedback economics. A historical ratio may be a starting heuristic; it is not the lesson's acceptance criterion.

“E2E is strongest because it is most realistic.” Realism is only one dimension. A broader boundary can observe a failure while providing weaker localization or a more fragile fixture. Evidence strength depends on fit to the claim, oracle, and seam.

“All tests are green, so the change is correct.” Green only means observed output matched defined expectations. If the expectation lacks authority or an important risk has no sensor, green results do not establish sufficiency.

“Acceptance test means E2E test.” Acceptance is a purpose/authority for an acceptability decision. If an approved rule can be supported at a service boundary, full E2E is not inherently required. Claims about actual UI or deployment, however, require those seams in the boundary.

When a shortcut appears, create a discriminating probe: What exact claim is being made? Which seam must execute? What authorizes the expected behavior? What blind spot remains?

Bridge from prior professional practice

If you already use a test pyramid, trophy, hourglass, or another portfolio model, keep it as a conversation tool rather than a law. Map the current portfolio row by row into the Risk-Based Test Strategy Record. Its existing distribution may be appropriate for your system; it may also be inherited convention.

For each suite, record purpose and boundary separately. Then identify oracle authority and the risk it addresses. Whenever the only rationale is “this test has always lived here” or “we must preserve this percentage,” the evidence chain is incomplete. Professional experience becomes reviewable when it can be translated into an inspectable trace.

The Fast Path skips explanatory repetition, not independent practice, mutation evidence, residual-risk disclosure, or the human decision.

Trade-offs: when should the boundary get broader?

A small boundary usually gives faster feedback and clearer localization, but observes fewer interactions. A broader boundary can provide more interaction realism, while setup cost, runtime, noise, and diagnosis often increase. The choice must follow the failure mode.

• For a pure invariant, a local sensor is usually the cheapest reliable choice.
• For a failure in the domain-to-persistence contract, integration is required because that seam is the claim.
• For a failure that only exists along a real multi-component path, a broader sensor is justified.
• For regression purpose, choose the boundary from the seam of the historical defect, not from the word “regression.”
• For acceptance purpose, record the approved authority and expand the boundary only enough to support the acceptance claim.

A strong portfolio does not necessarily contain more tests. It contains more explainable choices. A reviewer should be able to relate every added, moved, or removed sensor to a change in risk, seam, or maintenance economics.

Guided and independent practice

Guided practice

Take this possible defect: “the persisted total is stored under the wrong field name.” Before naming a test level, state the observable failure. Then identify the necessary seam: storage logic plus adapter contract. Bind the expectation to the adapter contract, build a controlled fixture, and select the smallest sensor that executes that seam. Finally state what it does not see, such as API-route wiring.

Independent practice

Now persistence moves behind a new boundary. API-wiring risk decreases, while timeout and contract-serialization risk increase. Redesign the Risk-Based Test Strategy Record. State which sensors remain, which move, and which are added or removed. No layer ratio is supplied.

Fade the scaffold before submission. In the final record, independently supply risk, observable failure, boundary, oracle authority, fixture, sensor, purpose, trade-off, and residual risk for every row.

What counts as reviewable evidence?

“I have a good strategy” is not evidence. The minimum reviewable package for this lesson includes:

1. A Risk-Based Test Strategy Record v1 tracing each material risk to a sensor and residual risk.
2. Deterministic execution of the local baseline fixture.
3. Independent sensors for local behavior, the persistence seam, the historical regression, and the acceptance rule where justified.
4. Mutation evidence showing sensitivity to targeted changes; “the suite is green” is not enough.
5. An explicit residual blind spot and a human decision to accept, modify, reject, or boundedly defer.
6. Disclosure of AI or tool support. Tools may propose test ideas, but the learner must independently defend sufficiency.

Lab TST001-LAB1 provides a small local service with no network dependency. The starter exposes risks, the service fixture, an approved acceptance rule, and a historical defect record. Mutation locations and the reviewer worked solution are not exposed. Multiple portfolios can pass if their evidence and limitations are defensible.

Assessment and transfer

S1 — retrieval: in a compact scenario, distinguish boundary, purpose, oracle, and residual risk. Use at least two lesson concepts without overclaiming.

S2 — application: build the central Risk-Based Test Strategy Record for a bounded case. Narrative alone is insufficient; each conclusion must have an inspectable evidence trail.

S3 — diagnosis: inspect a plausible portfolio in which one expected value was copied from current program output and one risk uses a sensor that never crosses the needed seam. Identify the first unsupported promotion and propose a probe that distinguishes competing explanations.

S4 — decision: choose ACCEPT, MODIFY, REJECT, or BOUNDED_DEFER from the available evidence. Cite evidence, limits, and at least one explicit non-claim. The final decision cannot be delegated to AI.

S5 — transfer: the service boundary and risk profile change in a nearby context. Redesign the portfolio and explain which sensors move, which remain, and why. Produce new evidence rather than copying old test paths.

The four critical rubric dimensions are claim-boundary/technical correctness, evidence traceability, human judgment with limitation disclosure, and transfer. An arithmetic average cannot compensate for failure on a critical dimension.

Sources, claim boundaries, and freshness

This lesson records three sourced/design claims with the following boundaries:

• TST-001-R01 — FACT: software testing has a foundational oracle problem. Academic authority: DOI:10.1109/TSE.2014.2372785. Retrieved 2026-09-02; review due 2027-09-02. This source does not prescribe a test framework or layer ratio.
• TST-001-R02 — FACT: two Google Testing articles discuss trade-offs among smaller/local, integration, and E2E tests: Just Say No to More End-to-End Tests and Fixing the Test Hourglass. Retrieved 2026-09-02; review due 2026-12-31. Historical portfolio shapes or numbers in that context are not MAKNOW requirements.
• TST-001-R03 — DESIGN: MAKNOW selects sensors from risk, seam, oracle, and decision needs and does not use a fixed test-layer percentage as an acceptance criterion. This is lesson design, not a universal empirical fact.

For this version’s source check, no registered source has crossed its review_due date. A practice-evolving source must be re-verified when its review date arrives before a later publication. The central uncertainty remains explicit: an optimal portfolio shape depends on system seams, risk, feedback economics, and infrastructure.