Saga Pattern and Compensating Transactions Explained

Quick answer

A Saga coordinates a workflow as local transactions. In an order → inventory → payment → notification flow, each service commits its own PostgreSQL state and an outbox intent. Publication is still at least once, so participants commit stable message IDs with their business effect.

When a later step cannot continue, the system issues compensating commands such as VoidPaymentAuthorization, ReleaseInventory, and CancelOrder. Compensation is a new, auditable business action, not a database rollback: earlier transactions already committed, concurrent work may exist, and a refund may differ from the original charge.

Use choreography for a short, stable event chain. Prefer orchestration when branching, deadlines, compensation, or human decisions require an explicit workflow owner. Either way, store progress durably, guard every transition, make forward and compensating handlers idempotent, and expose MANUAL_INTERVENTION when automation cannot restore a valid outcome.

Failure timeline

Consider Saga s-742 for order o-91:

1. The order service commits the order, Saga state INVENTORY_RESERVING, and a ReserveInventory outbox row together.
2. The relay publishes with broker confirmation, then crashes before marking the row published. Redelivery creates a duplicate command; inventory’s unique handler key makes it a no-op after the first reservation.
3. InventoryReserved advances the Saga to PAYMENT_AUTHORIZING and records AuthorizePayment.
4. Payment accepts the authorization but times out. The participant queries by the original idempotency key.
5. PaymentAuthorized advances to NOTIFICATION_SENDING; a stale InventoryReserved cannot cross its state and transition guards.
6. Notification times out, so the orchestrator reconciles by the same key instead of assuming failure.
7. Confirmed notification reaches COMPLETED; pre-pivot failure compensates; exhausted compensation reaches alerted MANUAL_INTERVENTION.

Durable state drives process restart.

Why one ACID transaction cannot span the workflow

A local ACID transaction can atomically change one service’s tables. It cannot normally include independently owned databases, a broker, and an email provider. Holding locks across remote calls creates contention without removing network ambiguity.

Two-phase commit requires compatible resources, adds availability coupling, and does not cover many APIs. A Saga accepts temporary inconsistency and defines valid terminal states.

Each participant still commits its state, processed-message marker, and outbox intent in one transaction. A Saga composes those invariants; it does not create a cross-service atomic commit or exactly-once delivery.

Choreography versus orchestration

In choreography, OrderPlaced triggers inventory, InventoryReserved triggers payment, and PaymentAuthorized triggers notification without a central controller. It suits a small linear flow. With more branches, cycles, timeouts, and compensation ownership become harder to see and test.

In orchestration, participants handle commands while an orchestrator interprets results. It owns sequencing and recovery policy, not participant data, making deadlines, pivots, and operator decisions visible.

Store orchestrator state in PostgreSQL and publish through an outbox. Do not let it update participant tables. Both need correlation and idempotency.

Model durable Saga state

Order creation has two failure boundaries. If the request fails before the Saga row commits, nothing is durable; the caller can retry the same start idempotency key. Once ORDER_CREATING commits, CreateOrderFailed is a durable result. Retry transient failures within a bounded budget. A permanent failure or exhausted budget reaches FAILED when no order exists. For an ambiguous result, reconcile by order_id: continue if the order exists, cancel it toward COMPENSATED if policy requires, or enter MANUAL_INTERVENTION if evidence remains unclear.

Use explicit states rather than a few booleans:

State	Meaning	Next action
ORDER_CREATING	Start request is durable; order creation is pending	Create order
INVENTORY_RESERVING	Order exists; reservation result is pending	Reserve inventory
PAYMENT_AUTHORIZING	Inventory is reserved	Authorize payment
NOTIFICATION_SENDING	Payment is authorized	Send notification
COMPENSATING_PAYMENT	Void the authorization	Then release inventory
COMPENSATING_INVENTORY	Release reserved units	Then cancel order
COMPENSATING_ORDER	Cancel the order	Finish compensation
COMPLETED	All required forward effects are confirmed	None
COMPENSATED	Required compensations are confirmed	None
FAILED	No forward business effect exists and creation is terminal	None
MANUAL_INTERVENTION	Automation stopped with an unresolved business state	Operator decision

These statuses can also be a semantic lock, a countermeasure in Microsoft’s Saga guidance. While the Saga is nonterminal, expose the order as PENDING and inventory as HELD_FOR_SAGA, not as final. Concurrent reads can display pending fulfillment; concurrent writes that would invalidate the held quantity must reject or defer; cancellation becomes a versioned Saga command rather than a direct row edit. This is a business-state rule, not a database lock held across services. COMPLETED confirms the order and consumes the hold. COMPENSATED or FAILED cancels the order where present and releases the hold. Commit each release and its outbox event atomically so a crash cannot leave the semantic lock orphaned.

Store facts needed to continue and audit:

CREATE TABLE order_saga (
  saga_id uuid PRIMARY KEY,
  order_id uuid NOT NULL UNIQUE,
  state text NOT NULL,
  version bigint NOT NULL DEFAULT 0,
  inventory_command_id uuid,
  reservation_id text,
  deadline_at timestamptz,
  last_error_code text,
  updated_at timestamptz NOT NULL DEFAULT now()
);

CREATE TABLE saga_transition (
  saga_id uuid NOT NULL REFERENCES order_saga,
  transition_id uuid NOT NULL,
  event_id uuid NOT NULL,
  from_state text NOT NULL,
  to_state text NOT NULL,
  occurred_at timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (saga_id, transition_id),
  UNIQUE (saga_id, event_id)
);

CREATE TABLE saga_event_receipt (
  saga_id uuid NOT NULL,
  event_id uuid NOT NULL,
  disposition text NOT NULL,
  PRIMARY KEY (saga_id, event_id)
);

The receipt intentionally omits a foreign key so a missing-Saga result can be deduplicated and routed for review.

Advance with an optimistic compare-and-set:

UPDATE order_saga
SET state = :next_state, version = version + 1, updated_at = now()
WHERE saga_id = :saga_id
  AND state = :expected_state
  AND version = :expected_version
RETURNING version;

Store inventory_command_id when enqueuing ReserveInventory, then store reservation_id when that effect is accepted. Zero returned rows means another transition won or the event is late; reload before deciding. In one transaction, persist the receipt and disposition, applied transition, and next outbox command. Uniqueness rejects repeats, while state + version prevents competing advances.

Design forward and compensating actions

Write a table before code:

Forward action	Success evidence	Compensating action
CreateOrder	OrderCreated with order ID	CancelOrder
ReserveInventory	Reservation ID and quantity	ReleaseInventory
AuthorizePayment	Provider authorization ID	VoidPaymentAuthorization
SendNotification	Provider message ID	Usually none; reconcile or repair

Every command carries saga_id, stable command_id, causation, business identifiers, and deadline. Results carry stable event_id and command identity. Participants enforce a business key plus message deduplication: (order_id, sku) for reservation and the command ID for payment.

Compensation uses recorded evidence: release the exact reservation and void the exact authorization. If it settled, policy may require a separately recorded refund.

Java Spring Boot orchestrator

This compact Spring service shows the transaction boundary, not a complete workflow engine. Repository SQL implements the guarded UPDATE, transition insert, and outbox insert:

@Service
public class OrderSagaOrchestrator {
  private static final Set<State> COMPENSATING = EnumSet.of(
      State.COMPENSATING_PAYMENT,
      State.COMPENSATING_INVENTORY,
      State.COMPENSATING_ORDER);
  private final SagaRepository sagas;
  private final OutboxRepository outbox;

  public OrderSagaOrchestrator(SagaRepository sagas, OutboxRepository outbox) {
    this.sagas = sagas;
    this.outbox = outbox;
  }

  @Transactional
  public void onPaymentFailed(ResultEvent event) {
    if (!sagas.recordEventOnce(event.sagaId(), event.eventId())) return;

    Saga saga = sagas.require(event.sagaId());
    if (saga.state() != State.PAYMENT_AUTHORIZING) return; // late or out of order

    int changed = sagas.advance(
        saga.id(), State.PAYMENT_AUTHORIZING,
        State.COMPENSATING_INVENTORY, saga.version());
    if (changed != 1) throw new OptimisticLockingFailureException("Saga raced");

    outbox.add(Command.withStableId(
        event.sagaId(), "ReleaseInventory", saga.reservationId()));
  }

  @Transactional
  public void onCompensationFailed(ResultEvent event) {
    if (!sagas.recordEventOnce(event.sagaId(), event.eventId())) return;
    if (!event.retryExhausted()) return; // the bounded retry policy still owns it

    Saga saga = sagas.require(event.sagaId());
    if (saga.state() != event.failedState() ||
        !COMPENSATING.contains(saga.state())) return; // delayed or terminal

    sagas.advanceOrThrow(
        saga.id(), saga.state(), State.MANUAL_INTERVENTION, saga.version());
    outbox.add(AlertCommand.forSaga(saga.id(), event.errorCode()));
  }
}

@Transactional protects this orchestrator’s PostgreSQL writes only. The outbox relay sends after commit and marks publication only after broker acceptance. A relay crash can duplicate the command, so each participant must use the idempotent consumer invariant. Delayed failures cannot reopen COMPLETED or COMPENSATED: the allowed source-state check and guarded update both reject them.

Node.js TypeScript orchestrator

The TypeScript version uses one checked-out PostgreSQL client because a transaction is connection-scoped:

import { randomUUID } from "node:crypto";
import type { Pool } from "pg";

async function onInventoryReserved(pool: Pool, event: ResultEvent) {
  const client = await pool.connect();
  try {
    await client.query("BEGIN");
    const receipt = await client.query(
      `INSERT INTO saga_event_receipt (saga_id, event_id, disposition)
       VALUES ($1, $2, 'RECEIVED')
       ON CONFLICT DO NOTHING RETURNING event_id`,
      [event.sagaId, event.eventId]
    );
    if (receipt.rowCount === 0) {
      await client.query("COMMIT"); // duplicate event
      return;
    }

    const loaded = await client.query(
      `SELECT state, version, inventory_command_id, reservation_id
       FROM order_saga
       WHERE saga_id = $1 FOR UPDATE`,
      [event.sagaId]
    );
    const current = loaded.rows[0] as
      { state: string; version: string; inventory_command_id: string | null;
        reservation_id: string | null } | undefined;
    if (!current) {
      await client.query(
        `UPDATE saga_event_receipt SET disposition = 'REVIEW_QUEUED'
         WHERE saga_id = $1 AND event_id = $2`,
        [event.sagaId, event.eventId]
      );
      await client.query(
        `INSERT INTO outbox_event
           (event_id, aggregate_type, aggregate_id, event_type, payload)
         VALUES ($1, 'OrderSaga', $2, 'ReviewSagaConflict', $3::jsonb)`,
        [randomUUID(), event.sagaId, JSON.stringify(event)]
      );
      await client.query("COMMIT");
      return;
    }
    const pastReservation = [
      "PAYMENT_AUTHORIZING", "NOTIFICATION_SENDING",
      "COMPENSATING_PAYMENT", "COMPENSATING_INVENTORY",
      "COMPENSATING_ORDER", "COMPLETED", "COMPENSATED",
      "FAILED", "MANUAL_INTERVENTION"
    ].includes(current.state);
    const sameCommand = current.inventory_command_id === event.commandId;
    const sameEffect = sameCommand && event.reservationId != null &&
      current.reservation_id === event.reservationId;
    const validActiveEvidence = current.inventory_command_id !== null &&
      event.commandId != null && sameCommand &&
      current.reservation_id === null && event.reservationId != null;
    if (pastReservation) {
      const repairable = [
        "COMPENSATING_PAYMENT", "COMPENSATING_INVENTORY",
        "COMPENSATING_ORDER", "COMPENSATED", "FAILED"
      ].includes(current.state);
      const repair = sameCommand && current.reservation_id === null &&
        event.reservationId && repairable;
      const disposition = sameEffect
        ? "DUPLICATE_EFFECT_IGNORED"
        : repair ? "REPAIR_QUEUED" : "REVIEW_QUEUED";
      await client.query(
        `UPDATE saga_event_receipt SET disposition = $3
         WHERE saga_id = $1 AND event_id = $2`,
        [event.sagaId, event.eventId, disposition]
      );
      if (!sameEffect) {
        const type = repair ? "ReleaseInventory" : "ReviewSagaConflict";
        await client.query(
          `INSERT INTO outbox_event
             (event_id, aggregate_type, aggregate_id, event_type, payload)
           VALUES ($1, 'OrderSaga', $2, $3, $4::jsonb)`,
          [randomUUID(), event.sagaId, type, JSON.stringify(event)]
        );
      }
      await client.query("COMMIT");
      return;
    }

    if (current.state !== "INVENTORY_RESERVING" ||
        Number(current.version) !== event.expectedVersion ||
        !validActiveEvidence) {
      await client.query(
        `UPDATE saga_event_receipt SET disposition = 'REVIEW_QUEUED'
         WHERE saga_id = $1 AND event_id = $2`,
        [event.sagaId, event.eventId]
      );
      await client.query(
        `INSERT INTO outbox_event
           (event_id, aggregate_type, aggregate_id, event_type, payload)
         VALUES ($1, 'OrderSaga', $2, 'ReviewSagaConflict', $3::jsonb)`,
        [randomUUID(), event.sagaId, JSON.stringify(event)]
      );
      await client.query("COMMIT");
      return;
    }

    const moved = await client.query(
      `UPDATE order_saga
       SET state = 'PAYMENT_AUTHORIZING', reservation_id = $3,
           version = version + 1
       WHERE saga_id = $1 AND state = 'INVENTORY_RESERVING'
         AND version = $2 RETURNING version`,
      [event.sagaId, event.expectedVersion, event.reservationId]
    );
    if (moved.rowCount !== 1) {
      await client.query(
        `UPDATE saga_event_receipt SET disposition = 'REVIEW_QUEUED'
         WHERE saga_id = $1 AND event_id = $2`,
        [event.sagaId, event.eventId]
      );
      await client.query(
        `INSERT INTO outbox_event
           (event_id, aggregate_type, aggregate_id, event_type, payload)
         VALUES ($1, 'OrderSaga', $2, 'ReviewSagaConflict', $3::jsonb)`,
        [randomUUID(), event.sagaId, JSON.stringify(event)]
      );
      await client.query("COMMIT");
      return;
    }

    await client.query(
      `INSERT INTO outbox_event
         (event_id, aggregate_type, aggregate_id, event_type, payload)
       VALUES ($1, 'OrderSaga', $2, 'AuthorizePayment', $3::jsonb)`,
      [event.nextCommandId, event.sagaId, JSON.stringify(event.payment)]
    );
    await client.query(
      `UPDATE saga_event_receipt SET disposition = 'APPLIED'
       WHERE saga_id = $1 AND event_id = $2`,
      [event.sagaId, event.eventId]
    );
    await client.query("COMMIT");
  } catch (error) {
    await client.query("ROLLBACK");
    throw error;
  } finally {
    client.release();
  }
}

A production implementation also validates schemas and retries serialization failures. The row lock stabilizes classification; the version remains a guard. Only the same stored command and reservation effect becomes DUPLICATE_EFFECT_IGNORED. A newly confirmed reservation from the timed-out command becomes REPAIR_QUEUED with exact ReleaseInventory; mismatched evidence becomes REVIEW_QUEUED. The terminal Saga state remains closed while the external effect is repaired. Active acceptance requires non-null matching command and reservation evidence; other conflicts and zero-row outcomes commit an intentional disposition.

Timeouts, duplicates, and late responses

A timeout is an event generated by a durable scheduler after deadline_at, not an in-memory timer. Claim timers with locking, record a unique timeout transition, and use the same guarded state change as any result.

A duplicate command is harmless only when the participant has already committed its processed marker and effect together. A duplicate event is harmless when the orchestrator’s (saga_id, event_id) uniqueness guard wins before transition logic. Out-of-order events must not skip states.

A process restart reloads rows and outbox work, never in-memory calls. After timeout compensation, compare a late success with the stored command and effect: ignore only the same effect, release a newly confirmed known reservation, and review conflicting identity. Never reopen a terminal Saga to repair an external leak.

Irreversible actions and pivot points

A pivot ends backward compensation. Identify it from business semantics: a legal notice, dispatched parcel, settled transfer, or email can be irreversible.

Place irreversible work after validation and compensable steps. In this example, order creation, inventory reservation, and payment authorization are compensable. Notification dispatch is the pivot. After NotificationSent, completion bookkeeping is retryable and idempotent; do not void payment because a later database acknowledgment was lost.

If payment capture is the real point of no return in your domain, move the pivot there and make notification a forward-recovery step. Document who can change that policy.

Compensation failure and manual intervention

Compensation is distributed work and can suffer rejection, timeout, duplicate delivery, and dependency outage. Retry transient failures with bounded backoff and the original compensation command ID. Route exhausted commands through the operated DLQ workflow; preserve identity and never replay around the normal handler.

When a compensation failure is permanent or ambiguous, atomically set MANUAL_INTERVENTION, store a redacted error, and enqueue an alert. The recovery record should show Saga and order IDs, current and intended states, participant evidence, attempted commands, deadlines, trace links, owner, approval policy, and an immutable operator log.

An operator may retry the same command, issue a separately authorized repair command, accept a documented business exception, or contact the customer. Recovery advances through the same version guard. Direct database edits erase evidence and can race a late event.

Testing strategy

Test the state machine as a transition table, then run PostgreSQL integration tests with real concurrent connections and a broker test environment:

• Order creation failure: before Saga commit, nothing persists; after ORDER_CREATING, transient CreateOrderFailed retries are bounded, then no-order cases reach FAILED, while ambiguous partial success reconciles before compensation.
• Inventory reservation failure: move from INVENTORY_RESERVING through COMPENSATING_ORDER to COMPENSATED; do not release nonexistent stock.
• Payment authorization failure: release the recorded reservation, cancel the order, and reach COMPENSATED.
• Notification failure before confirmed pivot: reconcile the provider; a confirmed permanent rejection moves to COMPENSATING_PAYMENT, while an ambiguous outcome retries with the same key or enters reviewed recovery.
• Duplicate command and duplicate event: one participant effect and one Saga transition commit.
• Active reservation: matching stored command with new valid reservation advances; absent, mismatched, or conflicting evidence queues review without advancing.
• Late reservation: the same stored command and reservation is ignored; a newly materialized reservation after compensation queues exact release; conflicting evidence queues review; terminal state stays unchanged.
• Semantic lock: concurrent reads see PENDING; edits and cancellation cannot bypass the Saga; completion consumes the hold and compensation releases it.
• Timeout: one scheduler wins; a competing result cannot cross the version guard.
• Process restart: pending state and outbox rows resume without reconstructing memory.
• Late success: reconcile and compensate its newly confirmed effect, never reopen a terminal Saga.
• Compensation failure: retry within budget, then enter MANUAL_INTERVENTION and alert once.

Also crash after every commit and before every acknowledgment. Cover concurrent conflicting results, out-of-order events, relay post-acceptance/pre-mark duplicates, participant rollback after marker insertion, and operator replay.

Monitoring and operations

Emit logs and traces with Saga, order, command, event, causation, and transition IDs, without payment secrets.

Track terminal outcomes, age by state, transition latency, timeouts, duplicates, conflicts, outbox lag, compensation age, DLQ depth, and MANUAL_INTERVENTION. Alert on stuck Sagas and growing compensation age; link the owning runbook.

Reconciliation compares Saga, participant, and provider records. It handles lost or late signals but does not replace reliable messaging.

Common mistakes

• Calling compensation a rollback or deleting audit history.
• Keeping the workflow only in memory.
• Publishing a command before the state transaction commits.
• Treating broker acknowledgment or a timeout as proof of the business outcome.
• Allowing state changes without expected state, version, and unique transition identity.
• Retrying irreversible work without a provider idempotency key or lookup.
• Assuming compensations cannot fail.
• Hiding unresolved cases in logs instead of a durable manual state.
• Letting the orchestrator write participant databases.

Practical checklist

• Draw forward, compensation, pivot, and terminal paths.
• Define success evidence and idempotency keys for every action.
• Persist explicit state, version, deadlines, errors, and transition history.
• Commit each local effect with its processed marker and outbox intent.
• Guard transitions by expected state and version; uniquely record events.
• Reconcile ambiguous timeouts and late results.
• Test every failed step, crash window, duplicate, and restart.
• Bound retries and connect DLQ recovery to the same handler.
• Alert on stuck compensation and provide approved manual repair.
• Monitor business outcomes as well as transport health.

Frequently asked questions

Is a Saga the same as a distributed transaction?

It composes local transactions into distributed business consistency; it is not one isolated ACID transaction.

Must compensation run in exact reverse order?

No. Dependencies and business risk determine order. Record prerequisites and allow safe parallel work.

Does orchestration create a single point of failure?

A durable, versioned Saga and outbox can restart and scale, but coordination remains critical infrastructure.

When should a team choose manual intervention?

Use it for conflicting evidence, irreversible actions, required approval, or exhausted compensation. Keep durable ownership and audit.

Sources

• Microsoft Azure Architecture Center: Saga distributed transactions pattern
• Microsoft Azure Architecture Center: Compensating Transaction pattern
• Spring Framework: Transaction management
• Spring Framework: Using @Transactional
• PostgreSQL: UPDATE
• PostgreSQL: Constraints
• PostgreSQL: Transaction isolation
• Node.js: crypto.randomUUID
• node-postgres: Transactions
• Microservices.io: Saga pattern

Related reading

• Continue through the System Design learning path and browse topic clusters.
• Build the publication boundary with Transactional Outbox Pattern Explained.
• Reason about ambiguous outcomes in Message Delivery Semantics Explained.
• Make handlers safe in Idempotent Consumer Pattern Explained.
• Operate exhausted recovery through Dead Letter Queue Explained.