Reliable consumption starts with the business effect, not with an annotation. Spring for Apache Kafka provides listener containers, acknowledgment modes, transactions, error handlers, retry topics, and dead-letter publishing. Each mechanism has a bounded contract that must match database and external-side-effect behavior.
Quick answer
Acknowledge or commit an offset only after the effect covered by that acknowledgment is durable. Assume a crash can cause redelivery, make handlers idempotent, classify failures before retrying, and send exhausted or permanent records to an operated dead-letter path. Non-blocking retry topics can change ordering, so choose them only when that trade-off is acceptable.
Prerequisites
Read Kafka Topics, Partitions, Consumer Groups, and Ordering, Message Delivery Semantics, and the Idempotent Consumer Pattern.
Define the durable effect
For a projection consumer, the effect may be one database transaction that inserts a deduplication marker and updates the projection. For an email consumer, a provider request and its durable delivery record may cross two systems. The listener container cannot infer which boundary makes the business outcome safe.
@Transactional
public void apply(OrderAccepted event) {
if (!processedEvents.insertIfAbsent("inventory", event.id())) return;
inventoryProjection.apply(event.orderId(), event.orderVersion());
}The uniqueness invariant and projection update must share the same database transaction. If the marker commits separately and the update rolls back, a retry can skip missing work.
ACK and offset decisions
Container acknowledgment modes determine when offsets become eligible for commit. Manual acknowledgment gives application code control but does not make external effects atomic with Kafka. Async acknowledgments can introduce additional in-flight behavior. Record the selected Spring Kafka version, mode, transaction manager, listener style, and batch behavior; test those exact settings rather than relying on a generic diagram.
When the listener participates in a Kafka transaction, a Kafka read-process-write sequence can receive exactly-once semantics within the documented scope. The Spring reference explicitly describes the read and process portions as at least once. A payment request or separate database is outside that broker transaction unless an independent contract handles it.
Blocking and non-blocking retries
A blocking retry pauses processing on the consumer path while an attempt is retried. It can preserve local order but may block unrelated records in the partition and threaten poll timing. Use short bounded retries only for failures likely to recover quickly.
Non-blocking retry publishes the record to retry topics with due times. It frees the main consumer but creates additional topics and consumers. Spring’s documented retry-topic pattern loses the original topic’s ordering guarantee. Do not adopt it for per-key state transitions unless version guards or another convergence strategy make reordering safe.
Dead-letter topic is an operated state
A DLT stores work that the automatic policy will not continue. Include original topic, partition, offset, stable event ID, schema version, bounded failure class, attempt count, and safe diagnostic references. Avoid copying secrets or raw exception messages into broadly accessible headers.
Replay requires authorization, a repaired cause, current business-state inspection, idempotency evidence, rate limits, and an audit trail. “Reconsume everything” is not a recovery plan.
Failure scenario
OrderAccepted version 5 fails because inventory is temporarily unavailable. A non-blocking retry moves it aside, and version 6 processes first. If the delayed version 5 later overwrites the projection, the system regresses. A conditional update such as “apply only when incoming version is greater than current version” converts reordering into convergence.
Common mistakes
- Retrying deserialization or validation failures that unchanged bytes cannot repair.
- Committing offsets after writing a deduplication marker but before the business update.
- Treating DLT publication as business completion.
- Exposing customer data and stack traces in retry headers or metric labels.
- Enabling non-blocking retry without testing order-sensitive consumers.
Production validation
Crash after database commit but before offset commit and verify safe redelivery. Stop the database, exhaust a bounded transient policy, then recover it and observe queue age and catch-up load. Send a poison record followed by a valid record. Test rebalance during processing, DLT publication failure, replay of an already completed event, and stale-version arrival. Monitor processing latency, lag, oldest age, retry rate, DLT age, deduplication outcomes, and business-state reconciliation.
Sources
- Spring for Apache Kafka reference 4.1, accessed 2026-08-18: https://docs.spring.io/spring-kafka/reference/kafka.html.
- Spring Kafka, “Exactly Once Semantics,” accessed 2026-08-18: https://docs.spring.io/spring-kafka/reference/3.3/kafka/exactly-once.html.
- Spring Kafka, “How the Retry Topic Pattern Works,” accessed 2026-08-18: https://docs.spring.io/spring-kafka/reference/retrytopic/how-the-pattern-works.html.
Related reading
Follow the complete Event-Driven Systems Learning Path. Continue with Dead Letter Queue Explained, Event Ordering and State Convergence, and Production Kafka Troubleshooting. Browse the topic cluster for every course lesson.