Event Schema Evolution Without Rewriting History
Evolve event schemas by preserving stored facts, registering versioned contracts, making compatible additive changes, and translating old payloads at read time when semantics change. Never rewrite production history merely to match newest application model.
Problem
Why this pattern exists
Events outlive services that wrote them. A field rename, unit change, split concept, or stricter required value may break projection rebuild years later even when live traffic appears healthy. Database migration that edits old event payloads damages auditability and makes historical evidence depend on mutable scripts.
Schema shape and event meaning are separate. Adding optional field is often structurally backward compatible; changing money from pounds to pence may keep JSON number type while changing semantics. Compatibility review needs examples, ownership, and replay tests—not validation keyword alone.
Design decisions
Make boundaries explicit
- 01
Prefer additive evolution
Add optional fields with deterministic defaults for old events. Keep old readers working during deployment overlap and avoid reusing a field name for different meaning.
- 02
Create new event for new meaning
When business fact changes materially, publish new event type or major schema version. Document relation to predecessor instead of forcing incompatible payload under familiar name.
- 03
Upcast at boundary
Translate historical representation into current in-memory shape before reducer. Upcaster must be deterministic, side-effect free, version-aware, and tested against captured fixtures.
AllSource implementation
Apply pattern to durable Core history
AllSource Core includes schema registry with subjects, versions, validation, and None, Backward, Forward, or Full compatibility modes. Register contract for each governed event type and validate producer payloads before acceptance. Tag schema versions and owners so operators can trace why a change was allowed.
Store original event unchanged. Consumer identifies schema or event version, runs required upcasters, then applies current reducer. Before release, replay representative histories—including oldest retained payloads—into new projection version. If semantic default cannot be derived from event itself, emit compensating enrichment fact rather than querying today's mutable external state during replay.
v1 order.placed { "total_pence": 14999 }
v2 order.placed { "total_pence": 14999, "currency": "GBP" }
read v1 -> default currency from recorded tenant contract version
read v2 -> use explicit currency
stored v1 payload remains unchangedFailure modes
Detect weak implementations early
New required field makes historical replay fail.
Fix: Make field optional with deterministic default or introduce new version and upcaster.
Same event type silently changes business meaning.
Fix: Create distinct fact or major contract version and document transition.
Migration rewrites old payloads and loses original evidence.
Fix: Preserve source event; translate on read or emit explicit correction event.
Production checklist
Ready when each statement is true
- Every governed event has owner and registered subject.
- Compatibility mode matches deployment and reader requirements.
- Semantic changes receive explicit version or event type.
- Upcasters are deterministic and covered by historical fixtures.
- Full projection replay passes before producer rollout completes.
Related patterns
Continue through adjacent decisions
Event replay
Event replay reads immutable events again in their original stream order and applies them to a new or reset consumer. Use replay to rebuild projections, reproduce historical state, test new reducers, or backfill derived outputs—never to re-trigger uncontrolled external side effects.
Read nextProjections and read models
A projection folds ordered events into a query-specific read model. It is derived state, not a second source of truth: operators must be able to discard it, replay source events, and build a replacement without rewriting event history or stopping existing readers.
Read nextAggregate streams
An aggregate stream is the ordered history for one consistency boundary, such as an order, account, or workflow. Give every aggregate a stable entity ID, append only facts owned by that boundary, and derive current state by folding its events in version order.
Read nextStore history once