Feature Flags, Canary and Bounded Rollout

1. The problem: deployment is not activation

A release can place a new artifact on a host while the new behavior remains disabled. A second control can enable that behavior only for a bounded audience. Treating deployment, activation, and audience exposure as one switch destroys the evidence needed to decide whether to progress safely.

The operational question is therefore not simply “is version 2 deployed?” It is: which artifact is present, what value did the feature evaluation resolve, for which subject or cohort, under which rule, during which observation window, and what health evidence supports the next action?

The artifact for this lesson is Bounded Rollout Control Record v1. It keeps those state classes separate so a rollout can end in PROMOTE, HOLD, ROLLBACK, or an explicit unknown rather than a vague “looks good.”

2. Feature flag evaluation is a runtime decision

A feature flag evaluation asks for a flag value at runtime. OpenFeature standardizes a vendor-neutral evaluation API: application code asks for a value, while a configured provider connects that request to a particular flag system. This means the application-side evaluation contract can remain stable even when the backing flag product changes.

Do not infer more than that. The evaluation API is not itself a universal flag-management control plane. Your control record therefore keeps the deployed artifact and the resolved flag state in different columns.

A safe row begins with at least: artifact version, deployment evidence, flag key, default value, resolved variant/value, evaluation reason when available, and the evaluation context used for targeting.

3. Targeting context must be reproducible

Targeting only helps if cohort membership can be reproduced. OpenFeature evaluation context supports contextual fields and an optional targeting key identifying the subject of evaluation. Providers can use this information for rule targeting, overrides, or fractional evaluation.

For a bounded rollout, the key property is determinism: the same synthetic subject under the same relevant context should stay in the same intended cohort. If a user moves between buckets because the targeting key changes format between requests, the percentage itself is meaningless.

Record the normalized targeting key, relevant non-sensitive attributes, rule identifier, bucket/cohort result, and enough evidence to repeat the classification. Personally identifying data is not required for this lesson; synthetic IDs are sufficient.

4. A percentage is an exposure limit, not a safety proof

A canary percentage limits how much of the population or workload receives the change. It does not prove the change is healthy. A 5% cohort with a feature-specific error can be unsafe; a 25% cohort with stable evidence can still require more bake time before promotion.

Progressive exposure reduces blast radius by expanding through successively larger groups. The control record must therefore name the current exposure ceiling and the evidence needed before the ceiling can increase.

Never write “10% passed, therefore safe.” Write “10% exposure observed for the required window; feature-specific health gate X passed; no recovery trigger fired; promotion policy permits the next bounded step.”

5. Separate canary deployment from feature activation

Canary deployment and feature flags solve related but different control problems. A platform can split traffic between old and new artifacts. A feature flag can separately decide whether behavior inside an artifact is active for a subject.

Possible states include:

• new artifact deployed to a canary, feature disabled for everyone;
• new artifact deployed broadly, feature enabled only for an internal cohort;
• two artifact versions receiving traffic while the new feature is active only in part of the new-version traffic;
• full artifact deployment with feature activation still blocked because health evidence is incomplete.

Bounded Rollout Control Record v1 records both deployment share and feature exposure when both exist. Do not collapse them into one “rollout percentage.”

6. Define the cohort before you evaluate health

Health data is interpretable only if you know who or what was exposed. Record cohort rule, targeting key policy, membership proof, exposure start, and exposure ceiling before judging feature health.

The cohort should be bounded by an explicit rule such as synthetic internal users, a stable hash bucket, a region, a service identity, or another reviewable attribute. The rule must be reproducible from the supplied evidence.

If cohort identity is unstable, halt progression even when aggregate service health is green. Otherwise the population measured by the health gate is changing underneath the decision.

7. Use feature-specific health gates

Deployment health and feature health can disagree. The process can be alive, latency can look normal globally, and the newly enabled path can still fail only for the exposed cohort.

A rollout gate should therefore identify feature-specific signals where possible: error rate for the activated path, latency for the affected request class, correctness probes, domain outcomes, saturation, or another relevant indicator. The gate needs a bounded comparison rule, not merely a dashboard color.

The record must preserve the observed signal and the decision derived from it. A green deployment summary cannot replace row-level evidence for the activated feature.

8. Bake time is evidence accumulation, not waiting for its own sake

A bake window gives slower or low-frequency failures time to appear. Its duration is a risk and observability choice, not a universal constant.

A short window can be green while a longer required window fails. In that case the rollout does not promote. Record: window start, required duration or completion criterion, observations collected, and whether the gate is complete.

If the required window has not completed, classify the decision as HOLD or UNKNOWN according to policy. Do not convert incomplete observation into a pass.

9. Flag-off is not always a complete rollback

Disabling a feature can stop future evaluations from entering the new path, but it may not reverse durable side effects. A request might already have written new data, changed schema state, sent an external message, or created an irreversible business action.

The recovery section must therefore ask two separate questions: “is activation disabled?” and “are side effects reconciled?” A kill switch can be effective while recovery is still incomplete.

Record the side-effect boundary, compatibility assumptions, compensating action if needed, and the condition that allows recovery to be called complete. If durable effects remain unresolved, the record must reject the statement “flag off = fully rolled back.”

10. Failure probe: unstable targeting key

Suppose the same synthetic subject is evaluated twice. In the first request its key is acct-0042; in the second, a middleware layer sends 42. Both represent the same logical subject, but the provider hashes different strings and places the subject in different percentage buckets.

This is a rollout-control failure even if both evaluations return valid flag values. Cohort assignment is no longer sticky, so exposure measurement is not reproducible. The correct decision is HOLD_TARGETING_UNSTABLE until key normalization is repaired and membership proof is repeatable.

The lesson intentionally treats this as more important than the nominal percentage. Deterministic cohort identity comes before percentage arithmetic.

11. Failure probe: deployment green, feature health red

A service can pass deployment probes while the enabled feature path fails for the canary cohort. This is the exact case that a feature-specific health gate must reveal.

If the supplied rollback trigger is met, choose ROLLBACK_REQUIRED and execute the bounded recovery path. If the trigger is not met but health still fails, choose HOLD_HEALTH_FAILED. Do not promote merely because the deployment process succeeded.

The distinction matters operationally: successful artifact delivery proves only that the new artifact reached the intended runtime location. It does not prove the activated behavior is safe.

12. Build Bounded Rollout Control Record v1 row by row

For each rollout stage, capture the following fields independently:

Keep direct observations, policy choices, and unknowns distinct. If a value is not observed, mark it unknown rather than filling it with a plausible guess.

13. Promotion, rollback, and retirement are different decisions

Promotion increases exposure. Rollback reduces or reverses exposure according to a defined recovery path. Retirement removes the temporary control after the rollout is stable and the flag is no longer needed.

A feature can be fully exposed but still not be ready for flag retirement. Examples: stale clients still need the default path, a prerequisite flag remains, cleanup code has not landed, or retirement verification is incomplete.

A retirement condition should name the owner, cleanup action, compatibility condition, and verification evidence. Leaving permanent flags without an owner creates stale-flag debt and makes future behavior harder to reason about.

14. Lab: analyze eight local rollout cases

The lab uses only synthetic local fixtures. Run the structural check, implement analyze_rollout(case), and classify eight evidence dimensions without seeing the expected answers.

The cases include: a healthy bounded cohort, an artifact deployed with the flag still disabled, unstable targeting, feature-specific health failure, durable side effects after flag disablement, a short green window followed by a failing required bake window, premature retirement, and a changed-context transfer case.

Your function must return deployment_state, targeting_state, cohort_state, activation_state, health_gate, bake_gate, recovery_state, and lifecycle_decision. The exercise rewards evidence separation rather than memorized labels.

15. Transfer: rebuild under a different cohort model

The transfer case changes subject identity, rollout stages, and the failing signal. Do not copy the first case's percentages or bake duration. Rebuild the control record from the new evidence.

What remains invariant is the method: prove deployment state, prove deterministic targeting, name the actual exposed cohort, evaluate feature-specific health, respect the required observation window, distinguish disabling from complete recovery, and define retirement only after stable full exposure plus cleanup evidence.

The final result should be reviewable by another engineer who can trace every decision back to observed evidence, an explicit policy choice, or a bounded unknown.