Schema Drift

1. What is the actual failure?

Schema drift appears when the shape emitted by a writer, the shape assumed by a reader, or the real projection in an environment moves outside the supported agreement. Difference is not the same as failure. A producer may add a nullable field that an older reader ignores; the schemas differ, yet the observed path can remain compatible. This lesson diagnoses drift from bounded evidence and the affected boundary rather than from the existence of a migration file or a textual diff.

2. Keep three projections separate

Build three independent views for every incident: writer schema, reader schema, and environment/runtime projection. The writer view says what is emitted or stored; the reader view says what is parsed or consumed; the environment projection shows the actual columns, types, constraints, or object shape present at the destination. Collapsing these views encourages premature closure. Schema Drift Failure Record v1 must identify where each observation came from.

3. PostgreSQL logical-replication boundary

PostgreSQL 18 documents that logical replication does not replicate schema definitions or DDL; published tables must exist on the subscriber and schema changes need separate coordination. Therefore, healthy data replication alone is not proof that publisher and subscriber schemas are synchronized. This claim is bounded to PostgreSQL logical replication rather than generalized to every replication product.

4. What pg_dump --schema-only proves

pg_dump --schema-only emits object definitions without table data or statistics. It can provide one bounded schema snapshot for comparison, but the snapshot does not prove application compatibility, successful rollout to every environment, or correctness of existing data under a new constraint. Treat it as an evidence input, not as closure.

5. Compatible versus incompatible drift

Compatibility is evaluated against an actual path. Adding a nullable field that an older reader ignores can be compatible. Renaming a required field, removing a depended-on field, introducing a non-convertible type change, or changing a constraint so that one environment rejects data accepted by another can be incompatible. Classification therefore needs the schema difference, reader/writer behavior, and reproduced failure evidence.

6. Counterexample: a new nullable field

The second fixture adds nullable nickname while the older reader explicitly ignores unknown fields. A learner who equates every schema diff with a breaking change fails this case. The difference is real, but no failure is reproduced on the observed path. Schema equality is not required for compatibility.

7. Required-field rename: a defensible break

The third fixture changes customer_id to account_id, while the reader still requires customer_id. The decode/validation failure is reproducible and the affected path is writer payload to reader decoder. Containment can pause rollout or preserve a compatibility alias. Repair needs an explicit order, such as teaching the reader to accept both names before the writer switches and removing the legacy form only after verification.

8. Constraint drift is more than text diff

Two environments can expose the same columns but enforce different constraints. If staging accepts NULL while another projection has NOT NULL on the same field, accepted-data behavior differs. The failure appears at the storage boundary. Column-name comparison is insufficient; constraints and accepted value domains are part of the schema projection.

9. Logical-replication schema lag

One fixture has a publisher that emits a new column while the subscriber still has the older projection. PostgreSQL does not automatically carry DDL through logical replication, so lag may remain hidden until incompatible data reaches the subscriber. Use reversible containment, synchronize the subscriber in a safe order, then rerun the same mutation. A resumed stream alone is not proof; the rerun oracle must exercise the repaired boundary.

10. Type drift and convertibility

A type difference is not automatically compatible or incompatible. Some paths can convert values safely; others cannot. If a reader requires an integer and the writer emits a non-numeric string, the failure is concrete. Record the value domain, conversion behavior, and reader requirement rather than applying a universal rule.

11. Containment is not repair

Containment limits blast radius: pause rollout, freeze the writer, route to a supported version, or temporarily block the mutation. Repair restores a supported boundary through reader-first migration, an additive compatibility window, environment synchronization, or mapper correction. Do not declare closure after containment. An independent rerun is still required.

12. Preserve OPEN when evidence is incomplete

If a writer snapshot exists but the reader snapshot or runtime projection is missing, do not invent the cause. Keep OPEN_BOUNDARY_UNRESOLVED, record the unknowns, and state the smallest probe needed to obtain the missing evidence. A professional diagnosis can end in a bounded unknown rather than false certainty.

13. Transfer: event payload plus database projection

The transfer replaces row-shaped data with event payload → mapper → database projection. Event v2 renames total_cents to amount_cents; the old mapper still reads total_cents, while a database column named total_cents can remain intentionally different by design. The drifting boundary is event to mapper, not necessarily the database naming. Rebuild the evidence chain instead of copying the row-based answer.

14. Lab and 64 assertions

The lab is fully local, deterministic, and synthetic. Eight fixtures are evaluated across eight dimensions: schema projection, reader/writer boundary, compatibility class, failure reproduction, affected path, containment, repair order, and rerun oracle. That produces 64 admin assertions. The point is disciplined evidence, not memorizing labels.

15. Exit checklist

To exit independently, separate writer, reader, and environment projections; name the exact drifting object; evaluate compatibility on the observed path; reject at least one weaker alternative with negative evidence; separate containment from repair; specify repair order; rerun the mutation after repair; and preserve OPEN when evidence is insufficient. The synthetic fixture makes no claim about any real production database.