P50/P80/P95 Forecasting

1) The real problem: three percentiles without context are not a forecast

P50/P80/P95 can look precise while being useless if the forecast object, horizon, data window, scenario, and distribution method are unknown. This lesson builds Percentile Forecast Evidence Pack v1, keeping each number beside the method and limitations that make it interpretable.

2) What a percentile says—and what it does not

Within a defined set/distribution, P95 is a threshold at or below which about 95% of observations fall. Google Cloud uses this interpretation for P95 latency within a measurement interval. That definition does not mean there is a 95% guarantee that future demand will never exceed the number. A forecast needs a future-oriented distribution or scenario process.

3) Name the forecast object precisely

“P95=820” is meaningless without an object. Is it five-minute peak requests per second, daily jobs, latency, or tokens per minute? The pack records forecast_object and unit so the percentile has a stable interpretation.

4) Horizon is part of the question

A one-month forecast and a one-quarter forecast are different problems. Seasonality, campaigns, trend, and evidence sufficiency can change. Changing horizon without refitting or revalidating produces HOLD; percentiles from the old horizon cannot simply receive a new label.

5) Keep sample window and sample size visible

A P95 computed from incomplete data—especially after removing high-demand observations—can shrink the tail artificially. The pack records the sample window, N, and completeness. If three weeks of data are later found incomplete, the previous result requires recomputation.

6) Record the distribution method

The lab uses an empirical nearest-rank percentile over sorted observations so the oracle is deterministic. Production may use a different method, but its name and scenario/version must remain attached to the pack. P50/P80/P95 without a generating method have incomplete provenance.

7) Treat tail drivers as evidence

A high P95 may be driven by a campaign, end-of-month batch, a specific tenant, or retries during failure. If a known spike exists in the calendar but is absent from the scenario, the pack must not remain USABLE. The tail driver is evidence, not decoration after the forecast.

8) Backtesting exposes historical error

A polished forecast without a backtest does not show how the method performed in comparable context. This lesson's MAKNOW DESIGN accepts backtest error up to 15%; higher error is INSUFFICIENT until the method is reviewed. That threshold is local design, not a Google Cloud requirement.

9) Freshness is its own state

If data freshness no longer matches the decision window, the pack becomes STALE even when its historical percentile calculation is correct. STALE means the old number may be valid for its old data, but it must be refreshed before the current decision.

10) Baseline: from observations to P50/P80/P95

The baseline contains 30 complete, fresh daily-peak observations, a one-month horizon under the current scenario, and 8% backtest error. Nearest-rank percentiles are computed from the visible sample and published with unit and tail driver. With object, horizon, sample, method, assumptions, and backtest present, the state is USABLE.

11) Wrong shortcut: delete the tail to make the forecast look cleaner

Removing several high-demand observations makes P95 lower and more attractive, but the declared sample is no longer represented. This mutation produces HOLD until the sample is repaired and the forecast rebuilt. A percentile is meant to expose distribution shape, not hide its tail.

12) Campaigns and known events belong in the scenario

A confirmed campaign announced after forecast generation changes the assumption set. If the spike is material and has not been incorporated, the state is HOLD. A separate campaign scenario is valid; presenting the no-campaign baseline as the campaign forecast is not.

13) Lab: eight fixtures, one reproducible oracle

The lab contains baseline, tail removal, horizon change, known-spike omission, missing object/unit, stale data, high backtest error, and transfer fixtures. The evaluator uses visible inputs and deterministic rules only. No answer key is stored under the learner path; evaluation uses visible inputs and explicit rules, while evaluation-only materials remain outside learner paths.

14) Transfer: campaigns, incomplete data, and a quarterly horizon

The transfer adds two known spikes, discovers three incomplete weeks, and changes the horizon from one month to one quarter. This is not merely a request for a new P95: the distribution process and evidence sufficiency changed. The state is HOLD until data repair, horizon refit, and spike scenarios are recomputed.

15) Delivery checklist

The evidence pack records Forecast ID/Object, horizon, sample window/N, seasonality/scenario, distribution method, P50/P80/P95, unit, tail driver, backtest error, data freshness, assumption set, decision use, and evidence ref. The decision must also state that a percentile is not a capacity guarantee, and material changes to horizon, sample, or scenario require recomputation.