Idempotent Event Consumers and Handlers
An idempotent event consumer produces the same durable result when it receives an event more than once. Use stable event IDs, entity versions, unique effect keys, and acknowledge only after processing succeeds; at-least-once delivery then becomes recoverable instead of corrupting state.
Problem
Why this pattern exists
Networks fail between committing work and acknowledging delivery. Consumer may finish update, lose connection, and receive same event again after restart. Trying to guarantee exactly-once transport across independent systems usually moves ambiguity rather than removes it. Idempotent business effects provide practical guarantee.
Different outputs require different guards. A projection can ignore an entity event whose version is not newer than last applied. A payment or email needs effect ledger keyed by stable event or command ID. An additive metric may need set membership or deterministic upsert instead of increment-on-delivery.
Design decisions
Make boundaries explicit
- 01
Choose deduplication key
Use immutable event ID for one effect per fact, command ID for one effect per request, or entity version for ordered projection state. Do not derive key from mutable payload fields.
- 02
Commit before acknowledge
Persist output and dedup marker in same transaction where target supports it. Then acknowledge event-store cursor. Crash before ack causes safe repeat; crash after ack leaves committed output.
- 03
Handle gaps explicitly
Per-entity version jump indicates missing or reordered input. Pause that entity, recover gap, then continue rather than accepting silently inconsistent projection state.
AllSource implementation
Apply pattern to durable Core history
AllSource durable consumers deliver committed events at least once. Core tracks acknowledged WAL position and replays from stored cursor after reconnect. ProjectionWorker adds per-entity version dedup as safety net, but cross-entity invariants and external effects still require application-level idempotency.
Give each deployed consumer version stable unique name, keep event-type filters narrow, and store processed event ID with outbound result. On reducer error, stop or dead-letter with enough event metadata to diagnose; do not advance cursor past an unhandled fact. Monitor repeat rate, version gaps, processing latency, reconnects, and checkpoint lag.
begin transaction
if processed_events contains event.id: return success
upsert invoice_status from event payload
insert processed_events(event.id, consumer = "billing_v2")
commit transaction
ack durable consumer positionFailure modes
Detect weak implementations early
Counter increments twice after consumer reconnect.
Fix: Use event-ID ledger or deterministic aggregate recomputation, not blind increment.
Cursor advances although target write failed.
Fix: Acknowledge only after durable target commit.
Two consumer versions share identity and move one cursor.
Fix: Version consumer IDs and run one owner per identity.
Production checklist
Ready when each statement is true
- Every effect has stable deduplication key.
- Output and processed marker commit atomically where possible.
- Cursor acknowledgement follows durable processing.
- Per-entity gaps and duplicates have explicit policy.
- Repeat delivery and checkpoint lag are measured.
Related patterns
Continue through adjacent decisions
Durable subscriptions
A durable subscription gives a named consumer a server-tracked position in the event log. After reconnect, event store replays committed events after last acknowledged position, then switches consumer to live delivery. Processing remains at least once, so handlers must be idempotent.
Read nextOptimistic concurrency
Optimistic concurrency protects a stream by accepting a write only when its expected version matches the current version. A mismatch means another command changed the aggregate first, so the caller must reload events, re-evaluate the command, and either append a new valid event or report a domain conflict.
Read nextEvent replay
Event replay reads immutable events again in their original stream order and applies them to a new or reset consumer. Use replay to rebuild projections, reproduce historical state, test new reducers, or backfill derived outputs—never to re-trigger uncontrolled external side effects.
Read nextStore history once