Message Delivery Semantics Explained

Quick answer

Delivery semantics describe what may happen when a producer, broker, network, or consumer fails around a message. At-most-once permits loss but avoids broker-driven redelivery. At-least-once avoids silent loss by retrying uncertain work, so duplicates are expected. Exactly-once is meaningful only when a named guarantee states the processing boundary and atomically covers every result inside it.

For most business workflows, choose at-least-once delivery, acknowledge only after durable state is committed, attach a stable message ID, and make the consumer idempotent. In the order → inventory → payment → notification flow, that means a repeated OrderPlaced must not reserve inventory twice, a repeated InventoryReserved must not charge twice, and a repeated PaymentCollected must not send duplicate receipts.

Broker terminology is specific. Apache Kafka consumers record offsets; RabbitMQ consumers acknowledge deliveries. Kafka transactions can atomically combine consumed offsets with records produced to Kafka topics. RabbitMQ manual acknowledgments tell RabbitMQ that a delivery was handled. Neither statement, by itself, makes a separate PostgreSQL commit, payment API, email send, or HTTP call happen exactly once.

Failure timeline

Trace one OrderPlaced through seven distinct moments:

A Kafka producer selects a partition and sends to its leader; a RabbitMQ publisher sends to an exchange.
The Kafka partition leader receives the selected record; the RabbitMQ exchange evaluates bindings and routes it to queues.
Kafka appends and replicates the record, while RabbitMQ stores or replicates it, to the configured durability level.
The producer receives a Kafka acknowledgment or RabbitMQ publisher confirm and any routing result.
The inventory consumer receives the message.
PostgreSQL commits the inventory reservation.
The consumer acknowledges the RabbitMQ delivery or commits its Kafka offset.

These stages are provider-specific. With Apache Kafka, acks=all waits for current in-sync replicas and, together with appropriate replication and min.insync.replicas, supports committed-log durability; acks=1 covers only the leader before follower replication, while acks=0 supplies no broker confirmation. With RabbitMQ, a publisher confirm can acknowledge an unroutable publish because the exchange resolved an empty queue list. A reliable publisher must use mandatory and handle basic.return, or an equivalent routing check, in addition to confirms. Recovery and confirm strength depend on durable exchange and queue declarations, persistent message delivery mode, and the selected classic, quorum, or stream replication behavior.

A failure before persistence can leave nothing to consume. A timeout after persistence but before producer confirmation is ambiguous: the broker may already own the message. The transactional outbox guide calls a confirmed send followed by a relay crash the post-acceptance/pre-mark duplicate window. On the consumer side, a crash before step 6 leaves no reservation. A crash after step 6 but before step 7 leaves a committed reservation but earlier broker progress. Redelivery is correct; the stable message ID must prevent another reservation.

Name the processing boundary

“Delivered exactly once” is incomplete. Ask: delivered from which component, processed into which result, and observed by whom?

A useful boundary might be “one Kafka input record, its output records in Kafka, and the corresponding consumed offset.” Kafka can cover that boundary with transactions and read-committed consumers. A different boundary might be “one message ID and one inventory row in PostgreSQL.” The application can cover that by recording the message ID and reservation in the same PostgreSQL transaction.

The boundary expands badly when the handler also calls unrelated systems. Email, HTTP, payments, and a separate database are external side effects unless a named system guarantee explicitly includes them. A database commit plus a successful HTTP response are not one atomic unit merely because they occur in one function. Document each boundary, owner, durable commit point, acknowledgment point, and recovery action.

At-most-once delivery

At-most-once means a message is processed zero or one time. It is appropriate when freshness matters more than completeness and replay has little value: disposable typing indicators, frequent telemetry samples, or cache hints whose next update repairs an omission.

On the consumer path, at-most-once behavior appears when progress is advanced before business processing. An Apache Kafka consumer can commit an offset before handling the record; a crash after the commit skips that record on restart. A RabbitMQ consumer using automatic acknowledgment lets RabbitMQ consider the delivery handled as it is sent; a connection or process failure can then lose work that never completed.

At-most-once does not prevent producer or upstream duplication; it only avoids recovering an uncertain delivery. It is usually unacceptable for inventory, payments, or the only notification of a critical event.

At-least-once delivery

At-least-once means a message should not be silently lost within the stated failure model, but it may be delivered or processed more than once. The consumer performs work first and records broker progress afterward. If it crashes between those operations, the broker repeats the delivery.

For Apache Kafka, the usual form is processing a record and committing its offset afterward. Kafka’s default guarantees are commonly described as at-least-once, but configuration and application code still matter. For RabbitMQ, manual acknowledgment after successful processing keeps an unacknowledged delivery eligible for automatic requeue when its channel or connection closes.

At-least-once trades an unrecoverable loss window for a controllable duplicate window. Use stable IDs, PostgreSQL uniqueness, business preconditions, bounded retries, and a terminal workflow. An in-memory “seen” set disappears on restart.

What exactly-once can and cannot mean

Exactly-once processing is possible inside a deliberately limited transactional boundary. Apache Kafka transactions can atomically publish output records to Kafka topics and update consumed offsets; consumers using isolation.level=read_committed avoid seeing aborted transactional output. Kafka Streams packages these mechanisms as exactly-once processing semantics for Kafka output topics and its state stores.

That guarantee does not automatically include an external PostgreSQL database, a card processor, an email provider, or an arbitrary HTTP endpoint. Kafka’s own design documentation says external destinations require cooperation. If a handler charges a card and then commits a Kafka offset, it can crash between those calls and charge again after redelivery.

Application-level “effectively once” behavior is the practical target. For inventory, store the message ID and reservation in one PostgreSQL transaction. For payments, use a documented provider idempotency key and persist its result. Email needs a durable ledger and provider-supported deduplication. Name the mechanism instead of advertising universal exactly-once delivery.

Producer confirmation and retry ambiguity

A successful local client call is not necessarily broker acceptance. Keep the Outbox article’s contract: publishing completes only after broker-confirmed acceptance at the selected durability level. Local buffering, enqueueing onto a framework executor, or writing bytes to a socket is insufficient.

Apache Kafka producers choose acknowledgment behavior with acks; idempotent producers use producer identity and sequence numbers to suppress duplicate log entries caused by retries. Kafka transactions can group writes to multiple Kafka partitions. These are Kafka-specific guarantees, and durability still depends on topic replication and broker configuration.

RabbitMQ publisher confirms are RabbitMQ’s separate producer-side mechanism. A confirm means the broker has taken responsibility according to the queue type, routing, persistence, and replication conditions described by RabbitMQ. Publisher confirms and consumer acknowledgments are orthogonal. A lost confirm leaves the publisher uncertain, so RabbitMQ recommends retransmitting unconfirmed publishes and designing consumers for duplicates.

Timeouts are not negative proof. Record stable message IDs before retrying and measure ambiguous outcomes. With a transactional outbox, retain the row until confirmation; after acceptance, mark it published. A crash between those actions preserves the post-acceptance/pre-mark duplicate window rather than risking message loss.

Consumer acknowledgment timing

Acknowledging before durable work creates at-most-once loss. Acknowledging after durable work creates at-least-once duplication. Acknowledging inside a finally block usually turns both transient and permanent failures into silent loss.

For RabbitMQ, set manual acknowledgment, process a bounded number of in-flight deliveries, and call basic.ack only after the local transaction commits. RabbitMQ automatically requeues unacknowledged deliveries when the channel or connection closes. A redelivered flag is a useful hint, not a substitute for a stable application message ID. Use negative acknowledgment deliberately: requeue transient failures with delay and limits; route terminal failures to an inspected dead-letter policy.

For Apache Kafka, disable automatic offset commits when handler completion must control progress. Commit offsets only after the durable effect succeeds, and handle partition revocation so work is not acknowledged by stale ownership assumptions. If PostgreSQL is the only output, advanced designs can store the source partition and offset in the same PostgreSQL transaction as the result, then seek from that durable position. That is an application protocol, not automatic Kafka-to-PostgreSQL atomicity.

Java Spring Boot acknowledgment example

This Spring Kafka example commits the PostgreSQL effect before acknowledging the Kafka record. The unique message_id is the durable duplicate barrier:

@KafkaListener(topics = "orders.placed", containerFactory = "manualAckFactory")
public void onOrderPlaced(OrderPlaced event, Acknowledgment ack) {
    inventoryService.reserveOnce(event); // returns only after PostgreSQL COMMIT
    ack.acknowledge();
}

@Transactional
public void reserveOnce(OrderPlaced event) {
    int inserted = jdbc.update("""
        INSERT INTO processed_message(message_id, consumer_name)
        VALUES (?, 'inventory')
        ON CONFLICT DO NOTHING
        """, event.messageId());
    if (inserted == 0) return;

    int updated = jdbc.update("""
        UPDATE inventory
        SET reserved = reserved + ?
        WHERE sku = ? AND available - reserved >= ?
        """, event.quantity(), event.sku(), event.quantity());
    if (updated != 1) {
        throw new IllegalStateException("missing SKU or insufficient stock");
    }
}

This is an Apache Kafka/Spring-specific acknowledgment API, not a cross-broker abstraction. The listener container must use a compatible manual mode. The row-count guard throws inside the transaction, so a missing SKU or insufficient stock rolls back the marker instead of acknowledging nonexistent work.

If the process crashes after PostgreSQL commits but before ack.acknowledge(), Kafka supplies the record again. ON CONFLICT makes that delivery a successful no-op. The pattern becomes unsafe if the reservation and marker use different transactions or databases.

Node.js TypeScript acknowledgment example

This RabbitMQ example uses manual acknowledgment and a PostgreSQL transaction. The call to channel.ack occurs only after COMMIT:

channel.consume("orders.placed", async (message) => {
  if (!message) return;
  let client: PoolClient | undefined;
  let transactionOpen = false;

  try {
    const event = parseOrderPlaced(message.content); // may throw InvalidMessageError
    client = await pool.connect();
    await client.query("BEGIN");
    transactionOpen = true;
    const marker = await client.query(
      `INSERT INTO processed_message(message_id, consumer_name)
       VALUES ($1, 'inventory')
       ON CONFLICT DO NOTHING`,
      [event.messageId],
    );
    if (marker.rowCount === 1) {
      const inventory = await client.query(
        `UPDATE inventory SET reserved = reserved + $1
         WHERE sku = $2 AND available - reserved >= $1`,
        [event.quantity, event.sku],
      );
      if (inventory.rowCount !== 1) {
        throw new InventoryUnavailableError(event.sku);
      }
    }
    await client.query("COMMIT");
    transactionOpen = false;
    channel.ack(message);
  } catch (error) {
    if (client && transactionOpen) {
      await client.query("ROLLBACK").catch(() => undefined);
    }
    const malformed = error instanceof InvalidMessageError
      || error instanceof SyntaxError;
    const requeue = !malformed && shouldRequeueWithinLimit(error, message);
    channel.nack(message, false, requeue);
  } finally {
    client?.release();
  }
}, { noAck: false });

This is RabbitMQ-specific behavior. parseOrderPlaced parses JSON and validates the schema; malformed content is terminal and is rejected without requeue for the configured dead-letter policy. shouldRequeueWithinLimit returns true only for a classified transient connection or processing failure below a durable attempt ceiling; production deployments should use a delayed retry topology rather than a hot loop. Parsing and connection acquisition are inside the error boundary, and optional-client cleanup settles failures even when PostgreSQL was never acquired. The row-count guard rolls back the marker on unavailable inventory. A crash after commit but before ack redelivers, while the existing marker preserves the successful reservation.

Decision table by workload

Workload	Sensible delivery choice	Duplicate control	Loss tolerance
Notifications	At-least-once for important receipts; at-most-once for disposable hints	Stable notification ID and send ledger	Product-specific
Analytics	Usually at-least-once; sometimes at-most-once for high-rate samples	Event ID, merge, or downstream aggregation	Often bounded
Payments	At-least-once command handling	Provider idempotency key plus durable local result	Near zero
Inventory	At-least-once	Message marker and reservation in one PostgreSQL transaction	Near zero
Cache invalidation	At-least-once or at-most-once when TTL repairs loss	Idempotent delete/version assignment	Usually bounded

Choose by workload and boundary, not broker brand. Payments and inventory need recoverability plus durable deduplication. Cache invalidation may accept loss when TTL repairs state. Notifications and analytics require explicit loss and duplicate policies.

Testing strategy

Use crash injection against real durable components, not only mocked acknowledgments:

Before processing: deliver OrderPlaced, terminate the consumer before BEGIN, restart it, and verify one eventual reservation.
After database commit: terminate immediately after PostgreSQL COMMIT but before broker progress, restart, and verify redelivery produces no second reservation.
Before acknowledgment: block or terminate on the line before Kafka offset commit or RabbitMQ ack; verify the same stable message ID returns and the consumer acknowledges the prior result.
Lose a producer confirmation after broker acceptance and verify the retry keeps the same event ID.
Make the broker reject a publish before acceptance and verify the outbox remains pending.
Race two consumer instances on one message ID and prove the PostgreSQL unique constraint permits one state change.
Inject a permanent validation error and prove retries are bounded rather than immediate forever.

Assert broker state, PostgreSQL rows, emitted follow-up events, and observable side effects. A passing handler unit test cannot establish crash semantics. Keep broker-specific integration suites because Kafka offset commits and RabbitMQ delivery acknowledgments fail differently.

Monitoring and operations

Measure unconfirmed publish age, lag or queue depth, oldest unacknowledged delivery, redelivery rate, duplicate conflicts, rollback rate, and terminal failures. Log message, causation, order, destination, and attempt identifiers.

Alert on sustained age and rate. Duplicate spikes can indicate crashes, acknowledgment timeouts, or retry storms. Runbooks should cover pausing, durable-state inspection, bounded replay, and downstream idempotency checks.

Common mistakes

Equating a broker acknowledgment with business success. Producer confirmation covers broker acceptance; consumer acknowledgment records delivery progress. Neither proves payment, email, HTTP, or another database completed exactly once.

Acknowledging in finally. Failed processing becomes permanent loss.

Generating a new ID on retry. Durable deduplication cannot connect attempts.

Using timestamps as identity. Clock precision and concurrency make them unreliable keys.

Deduplicating outside the business transaction. A crash can commit the marker without the effect, or the effect without the marker.

Trusting RabbitMQ’s redelivered flag as identity. It is delivery metadata, not an application uniqueness key.

Calling all Kafka processing exactly once. Kafka transactions cover named Kafka resources; external effects require explicit cooperation.

Infinite immediate requeue. Poison messages consume capacity and hide the underlying defect.

Practical checklist

Define producer, broker, consumer, durable effect, and acknowledgment boundaries.
Choose loss versus duplicate behavior per workload.
Require broker-confirmed acceptance before marking an outbox row published.
Reuse one stable message ID across every publish attempt and redelivery.
Commit business state before acknowledging.
Store the duplicate marker with the local effect in one PostgreSQL transaction.
Label Kafka- and RabbitMQ-specific behavior explicitly.
Treat email, HTTP, payments, and separate databases as external side effects unless a named guarantee includes them.
Bound retries and provide a terminal investigation path.
Test crashes before processing, after database commit, and before acknowledgment.
Monitor ambiguous publishes, redeliveries, duplicate conflicts, lag, and oldest age.
Document replay ownership and audit every manual replay.

Frequently asked questions

Is at-least-once the safest choice?

It is usually the safest transport choice for important work because it favors recovery over silent loss. It is safe for the business only when duplicate effects are controlled.

Does acknowledging after commit prevent duplicates?

No. It intentionally leaves a crash window after the commit and before acknowledgment. Durable idempotency makes the repeated delivery harmless.

Can Kafka guarantee exactly-once payment processing?

Not by itself. Kafka transactions can atomically cover Kafka records and consumed offsets. A payment provider is an external side effect unless a documented integration protocol, such as a provider idempotency key plus durable result storage, includes it.

Do RabbitMQ publisher confirms mean a consumer processed the message?

No. RabbitMQ documents publisher confirms and consumer acknowledgments as orthogonal mechanisms. A confirm covers the publisher-to-broker handoff, not downstream processing.

When is at-most-once reasonable?

Use it when missing one update is cheaper than repeating it and a later update, TTL, or recomputation repairs state. Record that loss tolerance explicitly.

Sources

Continue through the System Design learning path and browse topic clusters.
Close the producer dual-write gap with Transactional Outbox Pattern Explained.
Make repeated deliveries harmless with Idempotent Consumer Pattern Explained.
Route exhausted failures with Dead-Letter Queue Explained.