Production Database Engineering · Lesson 13

PostgreSQL and MySQL Replication and Failover Explained

Separate replication, apply, promotion, fencing, client routing, consistency, and ambiguous write evidence across both engines.

Quick answer

This lesson keeps the Spring order service as the application boundary.

Replication copies change; failover changes write authority. A safe order-service failover must establish one writer, fence the old writer, route clients, classify ambiguous transactions, restore protection, and verify durable business outcomes. Endpoint reachability alone proves none of those conclusions.

Learning objectives

  • Trace change generation, transport, replay, promotion, fencing, routing, and order verification.
  • Request PostgreSQL 18 and MySQL 8.4 evidence without pretending their log positions and topologies are identical.

Prerequisites

Know local transactions, replica lag, idempotency, connection pools, and backup recovery boundaries.

Shared relational contract

Define allowed data loss, interruption, read freshness, and ambiguous-write handling before an incident. Applications need finite connection retry, stable order identity, and a procedure for outcomes whose response was lost. Promotion must prevent simultaneous writers. Afterward, rebuild redundancy and retain evidence.

Synchronous acknowledgement can reduce some loss windows but changes availability and latency trade-offs; it does not replace application reconciliation or backups.

PostgreSQL 18 boundary

PostgreSQL physical streaming uses WAL locations and timelines; logical replication has different object and failover boundaries. Standby promotion creates a new write history. Operators must validate receiver, sender, replay, slots, archive continuity, timeline, and client routing using the deployed topology.

MySQL 8.4 InnoDB boundary

MySQL source-replica operation uses binary logs, relay/apply state, positions or GTIDs according to configuration. Promotion, semi-synchronous behavior, replication filters, and topology management require 8.4-specific procedures. Group Replication is a separate optional architecture, not the assumed core contract.

Production failure scenario

The primary becomes unreachable while several checkout requests are committing. A standby is promoted, but the old host later returns. Fence it before accepting writes, establish the authoritative history, and classify timed-out orders through idempotency and durable records. Then verify new writes, replicas, outbox flow, and affected customers.

Common misconceptions

  • A DNS change is not a complete failover.
  • Zero reported lag does not prove every order is fresh.
  • Replication does not replace isolated backups.
  • Retrying an ambiguous write without identity can duplicate it.

Decision checklist

Document connection draining, promotion authorization, client caching, backup continuity, and rejected-request reconciliation before reopening ordinary traffic after an interruption. Retain the exact topology and release identity.

  • Name write authority and fencing owner.
  • Record application and log positions.
  • Bound retries and connection storms.
  • Classify ambiguous orders before replay.
  • Rebuild redundancy after promotion.
  • Verify durable state and customer outcomes.

Review backup and restore, continue to sharding, use the course and topic, deepen with production slow-query troubleshooting, and connect to production resilience.

Official sources

Knowledge check

Check your understanding

Answer both questions correctly to mark this lesson as mastered. You can retry without penalty.

1. The old primary becomes reachable after promotion; what must happen before accepting writes?

2. A failover endpoint is reachable; when is the service recovered?