Reliable brokers can still deliver duplicates, and parallel systems can still expose records in an order that differs from business causality. Correct consumers use stable identity and versioned state transitions so repeated or stale work converges instead of corrupting state.
Quick answer
Use one stable event ID to detect the same logical event, an aggregate or entity version to reject stale state, and a database invariant to make the decision atomic. Partition keys can preserve order within a broker partition, but consumers still need convergence rules for retries, replay, producer races, and migrations.
Prerequisites
Read Idempotent Consumer Pattern Explained, Kafka Topics and Ordering, and Eventual Consistency Explained.
Identity and version solve different problems
eventId answers whether this is the same logical event seen before. orderVersion answers whether this event describes newer state than the projection already contains. A duplicate retains both values. A later event has a new ID and a greater version. A retry that generates a new event ID defeats duplicate detection.
Do not use payload equality as event identity. Two legitimate BalanceChecked events can carry identical data, while one event may be serialized differently by two producers. Persist identity intentionally.
Atomic duplicate handling
A consumer can create a uniqueness constraint on (consumer_name, event_id). Insert the marker and apply the business update in one transaction. Concurrent deliveries race at the database invariant; one commits and the other observes the duplicate.
BEGIN;
INSERT INTO processed_event (consumer_name, event_id)
VALUES ('order_projection', :event_id)
ON CONFLICT DO NOTHING;
UPDATE order_projection
SET status = :status, source_version = :version
WHERE order_id = :order_id
AND source_version < :version;
COMMIT;Application code must check whether the insert and update actually affected rows. The example shows two guards, not a complete handler. If the marker inserts but the version update is stale, the handler should intentionally record the disposition rather than claim it applied new state.
State versus delta events
A state event says “order version 7 is PAID.” A stale version can be ignored after a conditional update. A delta event says “increment balance by 5.” Applying it twice corrupts state, while dropping it loses value. Delta consumers need stronger deduplication and often sequence-gap detection.
Commutative operations can converge in any order only when their algebra and business limits allow it. “Add item to a set” may commute; “charge card” does not. Do not invoke CRDT terminology as a substitute for defining the actual invariant.
Ordering domains
Choose the smallest key that needs serial history. orderId may be sufficient for order state, while inventory availability spans many orders for the same SKU. One key cannot automatically preserve every cross-aggregate invariant. Those decisions belong in the owning service and database transaction, not in a consumer’s hope that global event order exists.
Consumer parallelism can reorder completion even when poll order is stable. If record 2 performs a slow call while record 3 finishes quickly on another thread, the applied order may differ. Bound concurrency per ordering key or use conditional versions that make completion order harmless.
Failure scenario
Version 12 enters a retry topic while version 13 remains on the main topic and commits first. Version 12 later returns. A blind upsert regresses the customer status. A conditional WHERE source_version < 12 affects zero rows, and the handler records STALE_IGNORED without repeating an external side effect.
Common mistakes
- Using broker offset as a global business version.
- Deduplicating in an in-memory cache that disappears during failover.
- Persisting the deduplication marker separately from the business change.
- Assuming one timestamp establishes causal order across services.
- Replaying a projection stream through live notifications or payments.
Production validation
Deliver the same event concurrently, deliver versions in reverse order, leave a sequence gap, restart after the effect but before ACK, and replay historical records. Assert the final projection, deduplication disposition, external-side-effect count, and audit evidence. Monitor duplicates, stale ignores, gaps, version regressions, reconciliation mismatches, and the oldest unresolved gap. A rising duplicate count can reveal relay or acknowledgment problems even when state remains correct.
Sources
- Apache Kafka 4.1 design documentation, accessed 2026-08-18: https://kafka.apache.org/41/design/design/.
- CloudEvents Specification repository, accessed 2026-08-18: https://github.com/cloudevents/spec.
Related reading
Use the Event-Driven Systems course and topic cluster for the complete sequence. Continue with Reliable Spring Kafka Consumers, Saga Pattern and Compensating Transactions, and Production Kafka Troubleshooting. This lesson also deepens the broader System Design path.