A Spring service cannot atomically commit a relational database transaction and an ordinary Kafka send merely by placing both calls in one Java method. The transactional outbox makes publication intent part of the local database commit, then uses a recoverable relay to publish it.
Quick answer
Insert the business change and an outbox row in the same local transaction. Publish committed rows with a bounded relay, retain one stable event ID across attempts, and mark publication only after broker acceptance. Expect duplicate sends when the relay crashes after acceptance but before recording success, so consumers still need idempotency.
Prerequisites
Read Transactional Outbox Pattern Explained, Spring Service Layers and Transaction Boundaries, and Event Design with CloudEvents and AsyncAPI.
One local commit
The application service owns the local business invariant:
@Transactional
public OrderId acceptOrder(AcceptOrder command) {
Order order = orders.insert(command);
outbox.insert(OutboxEvent.orderAccepted(
eventIds.next(), order.id(), order.version(), clock.instant()
));
return order.id();
}If either insert fails, the database transaction rolls back both. The method does not claim that Kafka accepted the event. It proves only that the order and its durable publication intent committed together.
The outbox row should contain a stable event ID, aggregate ID, event type, schema version, payload or a reproducible snapshot, creation time, attempt metadata, and publication state. Keep secrets and unnecessary personal data out of the payload.
Relay ownership
A polling relay can select a bounded batch. Holding FOR UPDATE SKIP LOCKED locks through a slow broker call is simple but lengthens transactions. An alternative is an atomic short transaction that claims rows with an expiring lease, commits, publishes outside the database transaction, and records broker-confirmed success in another short transaction. The lease must recover after crashes without allowing stale workers to overwrite current ownership.
Change-data-capture can publish committed outbox rows without an application poller, but it shifts ownership to the database log, connector, broker, and their checkpoint contracts. It does not eliminate duplicates or schema design.
Kafka producer boundary
KafkaTemplate.send returns an asynchronous result. Treat completion according to the selected Spring Kafka version and producer configuration. A callback invocation before the future completes is not durable evidence. Broker acknowledgment also does not mean a consumer’s database or payment API changed.
Kafka producer idempotence and transactions can protect specific producer and Kafka read-process-write boundaries. They do not atomically include an unrelated relational database plus an arbitrary external service. Keep the outbox and consumer idempotency contracts explicit even when Kafka transactions are used elsewhere.
Stable identity across attempts
If relay attempt two generates a new event ID, consumers cannot recognize it as the same logical event. Persist the ID in the outbox row and send it on every retry. Publication attempt IDs may change for diagnostics, but they must not replace business event identity.
Use the aggregate ID as the Kafka key when that aggregate needs per-partition order. Event ID and partition key solve different problems: one supports duplicate recognition, while the other selects an ordering domain.
Failure scenario
The relay publishes OrderAccepted, Kafka acknowledges it, and the process dies before updating the outbox row. After lease expiry, another relay publishes the same event ID. This is a correct at-least-once recovery outcome. A consumer with a uniqueness constraint on (consumer_name, event_id) converges; a consumer that sends email before durable deduplication may notify twice.
Common mistakes
- Publishing directly after a database commit and losing the event when the process dies.
- Marking an outbox row published before broker acceptance.
- Generating payloads later from mutable business rows and changing event meaning.
- Retrying forever without backoff, age limits, or operator visibility.
- Deleting rows immediately and losing reconciliation evidence.
Production validation
Inject crashes before commit, after commit, before broker send, after broker acceptance, and before publication-state update. Run two relays concurrently and verify claims do not lose rows. Measure unpublished count, oldest unpublished age, attempts, claim expiry, publish latency, failures by bounded class, and reconciliation between committed outbox IDs and broker observations. Test broker outage recovery without overwhelming Kafka or the database.
Sources
- Spring for Apache Kafka reference, accessed 2026-08-18: https://docs.spring.io/spring-kafka/reference/kafka.html.
- Apache Kafka design documentation, accessed 2026-08-18: https://kafka.apache.org/41/design/design/.
Related reading
Continue through the Event-Driven Systems course with Message Delivery Semantics, Reliable Spring Kafka Consumers, and the Idempotent Consumer Pattern. The complete cluster is at Event-Driven Systems topics.