Replication

Solutions

Database Monitoring Software Without Query Languages

Database monitoring for MySQL, PostgreSQL, SQL Server, Oracle & MongoDB with per-second query insights, deadlock detection & AI root cause. Try free today!

Guides

Guides overview 5 pages

ClickHouse Operations Guides

Practical guides for running, troubleshooting, and monitoring ClickHouse in production.

CockroachDB Operations Guides

Practical guides for running, troubleshooting, and monitoring CockroachDB in production.

MongoDB Operations Guides

Practical guides for running, troubleshooting, and monitoring MongoDB in production.

MySQL Operations Guides

Practical guides for running, troubleshooting, and monitoring MySQL in production.

PostgreSQL Operations Guides

Practical guides for running, troubleshooting, and monitoring PostgreSQL in production.

Cassandra Operations Guides 6 pages

Open the Cassandra Operations Guides hub

Cassandra hint overflow: max_hint_window expiry and silent data divergence

Diagnose and recover from Cassandra hint window expiration, hint accumulation, and the silent data divergence that follows extended replica outages.

Cassandra hints accumulating: hinted handoff backlog and replay storms

Diagnose and resolve Cassandra hinted handoff backlog and replay storms before coordinator disk exhaustion and cascading replica failures.

Cassandra node showing DN in nodetool status: gossip, phi, and recovery

Diagnose why Cassandra nodes appear as DN in nodetool status, understand phi accrual failure detection, and recover from gossip flapping and network partitions.

Cassandra repair not running: the silent gap that resurrects deleted data

Detect when Cassandra anti-entropy repair stalls, diagnose why repair fails to complete within gc_grace_seconds, and prevent silent tombstone resurrection.

Cassandra UnavailableException: not enough replicas for the consistency level

Diagnose and recover from Cassandra UnavailableException when the coordinator cannot satisfy the requested consistency level due to insufficient live replicas.

Cassandra zombie data resurrection: gc_grace_seconds and unrepaired tombstones

Deleted data reappears when tombstones expire before repair reconciles them across replicas. Detect the silent failure and recover safely.

Ceph Operations Guides 5 pages

Open the Ceph Operations Guides hub

Ceph degraded objects: reduced redundancy and the race against a second failure

How to read ceph_num_objects_degraded correctly, distinguish it from misplaced objects, and alert on stalled recovery instead of normal healing.

Ceph PG degraded: fewer replicas than the pool size, and when it matters

What a degraded Ceph placement group means, when it is normal during recovery, and how to alert on it without flooding tickets.

Ceph PG down: no surviving replica for reads or writes

Diagnose and recover Ceph placement groups stuck in the down state, where no surviving replica can serve reads or writes.

Ceph PG incomplete: placement groups that cannot serve I/O

What a Ceph PG in incomplete state means, how to diagnose peering failures with ceph pg query, and the recovery paths from OSD restart to force-create-pg.

Ceph PG inconsistent (OSD_SCRUB_ERRORS): scrub found replica divergence

Ceph scrub found replica divergence. Diagnose which copy is corrupt before running ceph pg repair, or you may propagate the corruption.

ClickHouse Operations Guides 12 pages

Open the ClickHouse Operations Guides hub

ClickHouse cannot connect to ZooKeeper/Keeper: diagnosing the coordination layer

Diagnose and recover when ClickHouse loses its ZooKeeper or ClickHouse Keeper coordination layer, including connection checks, 4lw probes, and replica readonly states.

ClickHouse checksum mismatch and broken parts: detecting data corruption

Detect and remediate ClickHouse data corruption from checksum mismatches, broken parts, and detached parts before replication divergence spreads.

ClickHouse distributed DDL stuck: ON CLUSTER queries that never finish

Diagnose and fix ON CLUSTER DDL queries that hang or never finish in ClickHouse. Identify stuck hosts, reconcile schema drift, and restore coordination with ZooKeeper or Keeper.

ClickHouse Keeper saturation spiral: too many tables, DDL storms, and cluster freeze

Diagnose and break the ZooKeeper/Keeper saturation spiral that turns replicated ClickHouse clusters read-only.

ClickHouse merge death spiral: when parts accumulate faster than merges consolidate

Break the feedback loop where ClickHouse inserts outrun background merges, parts pile up, and writes are delayed or rejected.

ClickHouse monitoring checklist: the signals every production cluster needs

A production monitoring checklist for ClickHouse organized by maturity level, from survival signals to expert diagnostics.

ClickHouse mutation stuck: parts_to_do not decreasing and how to recover

Diagnose and recover stuck ClickHouse mutations when parts_to_do stops decreasing, including quick checks, root causes, and safe remediation steps.

ClickHouse Replica is lost: SYSTEM RESTORE REPLICA and recovering a diverged replica

Diagnose and recover a ClickHouse replica that has silently diverged from its peers using SYSTEM RESTART REPLICA and the supported limits of SYSTEM RESTORE REPLICA.

ClickHouse replication lag: absolute_delay, queue_size, and catch-up diagnosis

Diagnose ClickHouse replication lag using system.replicas, system.replication_queue, and background pool signals. Learn to tell a saturated fetch pool from a stuck mutation or a readonly replica.

ClickHouse replication queue stuck: num_tries, last_exception, and dead entries

Diagnose and fix stuck ClickHouse replication queue entries by reading num_tries, last_exception, and entry types instead of relying on delay alone.

ClickHouse Table is in readonly mode: is_readonly on replicated tables and how to fix it

Diagnose and recover replicated ClickHouse tables stuck in readonly mode caused by lost Keeper sessions, coordination failures, or quorum loss.

ClickHouse ZooKeeper session has expired: causes, recovery, and tuning

Diagnose and recover when ClickHouse replicas lose ZooKeeper sessions, become read-only, and stop accepting writes.

CockroachDB Operations Guides 3 pages

Open the CockroachDB Operations Guides hub

CockroachDB consistency check failed: checksum mismatches and confirmed corruption

Operational reference for diagnosing and responding to CockroachDB consistency check failures, checksum mismatches, and confirmed data corruption.

CockroachDB replica unavailable: lost quorum and stuck Raft groups

Diagnose and resolve CockroachDB replica unavailable errors, lost Raft quorum, and stuck Raft groups causing range unavailability.

CockroachDB under-replicated ranges: ranges_underreplicated and the healing margin

Diagnosing CockroachDB under-replicated ranges, understanding the healing margin, and configuring alerting that separates expected recovery from dangerous fault tolerance loss.

Elasticsearch Operations Guides 2 pages

Open the Elasticsearch Operations Guides hub

Elasticsearch ALLOCATION_FAILED after max retries: reroute and corrupt shard recovery

Recover Elasticsearch shards stuck in ALLOCATION_FAILED after exceeding max retries, including reroute API usage and corrupt shard recovery paths.

Elasticsearch cluster health yellow: unassigned replicas vs real allocation blocks

Distinguish benign yellow cluster states from real allocation blockers. Diagnose unassigned replicas, disk watermarks, and shard allocation failures in Elasticsearch.

Kafka Operations Guides 15 pages

Open the Kafka Operations Guides hub

Kafka FailedProduceRequestsPerSec rising: the single best 'producers are hurting' signal

Diagnose rising Kafka FailedProduceRequestsPerSec, correlate with ISR and replication health, and stop producer-visible failures before they cascade.

Kafka fetch request latency high: FetchConsumer vs FetchFollower and page cache misses

Diagnose elevated Kafka fetch request latency by distinguishing consumer fetches from replica fetches and identifying page cache misses as the root cause.

Kafka ISR shrinking: IsrShrinksPerSec, flapping, and the cascade to offline

Diagnose ISR shrink storms, flapping replicas, and the cascade to offline partitions in Kafka production clusters.

Kafka LEADER_NOT_AVAILABLE: causes during elections, restarts, and topic creation

Diagnose transient and persistent Kafka LEADER_NOT_AVAILABLE errors by correlating leader elections, controller health, ISR state, and offline partitions.

Kafka Log directory failed / OfflineLogDirectoryCount > 0: disk errors and JBOD recovery

Troubleshoot Kafka brokers with offline log directories due to disk I/O errors, JBOD failures, and recover partition availability without data loss.

Kafka min.insync.replicas and acks: configuring durability you actually have

Why acks=all without min.insync.replicas only guarantees leader durability, and how to configure Kafka replication for real fault tolerance without blocking writes.

Kafka NOT_LEADER_FOR_PARTITION: stale metadata, controller lag, and client retries

Diagnose persistent NOT_LEADER_FOR_PARTITION errors by distinguishing transient leader elections from controller event queue backup and broker metadata desync.

Kafka NotEnoughReplicasException: acks=all writes rejected below min.insync.replicas

Diagnose and fix Kafka NotEnoughReplicasException when acks=all writes fail because the ISR shrank below min.insync.replicas.

Kafka OfflinePartitionsCount > 0: partitions with no leader and how to recover

Diagnose and recover Kafka partitions with no active leader when OfflinePartitionsCount is nonzero.

Kafka OffsetOutOfRangeException: when retention deletes data before the consumer reads it

Fix and prevent Kafka OffsetOutOfRangeException when consumer lag outruns log retention, causing data loss or forced offset resets.

Kafka replica MaxLag growing: slow followers and replica fetcher health

Diagnose and fix growing Kafka replica fetcher lag before it triggers ISR shrinks and write rejections.

Kafka REQUEST_TIMED_OUT: produce requests that expire before replication completes

Diagnose and fix Kafka REQUEST_TIMED_OUT errors when acks=all produce requests expire waiting for replication, request queue backup, or slow disk.

Kafka UncleanLeaderElectionsPerSec > 0: confirmed silent data loss

What to do when Kafka's UncleanLeaderElectionsPerSec metric rises above zero, confirming silent data loss, and how to recover without deepening the damage.

Kafka UnderMinIsrPartitionCount: confirming the write path is blocked

How to read Kafka's UnderMinIsrPartitionCount metric, distinguish it from under-replication, and confirm producers are actively blocked.

Kafka UnderReplicatedPartitions > 0: the most important metric and how to clear it

Troubleshoot and clear Kafka under-replicated partitions by identifying lagging followers, correlating disk, network, and GC signals, and applying the right fix without causing further ISR shrink.

Microsoft SQL Server Operations Guides 3 pages

Open the Microsoft SQL Server Operations Guides hub

SQL Server AG send and redo queues growing: replication lag and failover RTO

Diagnose growing log_send_queue_size and redo_queue_size in Always On Availability Groups, estimate failover RTO, and resolve replication lag.

SQL Server Availability Group not synchronizing: NOT_HEALTHY replicas and failover risk

Diagnose and resolve SQL Server Availability Group replicas stuck in NOT_HEALTHY or DISCONNECTED states, and understand when failover capability is actually compromised.

SQL Server Error 9002: the transaction log for the database is full

Diagnose and fix SQL Server error 9002 by reading log_reuse_wait_desc first, then resolving the specific blocker: log backups, open transactions, AG lag, or replication.

MongoDB Operations Guides 11 pages

Open the MongoDB Operations Guides hub

MongoDB connection storm spiral: reconnection floods after an election or deploy

Diagnose and break MongoDB connection storm spirals caused by elections, deploys, or network blips. Recognize RSS spikes, ticket contention, and totalCreated churn before they crash the node.

MongoDB disk full: emergency recovery when mongod can't write the journal

Emergency triage and recovery steps when MongoDB exhausts disk space and the journal cannot write, including WiredTiger space reclamation and recovery procedures.

MongoDB monitoring checklist: the signals every production cluster needs

A leveled checklist of production MongoDB monitoring signals covering liveness, replication, WiredTiger cache, tickets, connections, and disk saturation.

MongoDB no primary / election storm: repeated elections and write outages

Diagnose and stop MongoDB replica set election storms where primaries repeatedly step down, causing rolling 2-12 second write outages.

MongoDB not master error: writes hitting a non-primary node after failover

Troubleshoot and fix MongoDB NotWritablePrimary errors that occur when applications write to a non-primary node after failover or election.

MongoDB not primary and secondaryOk=false: reading from a secondary and how to fix it

Fix MongoDB NotPrimaryNoSecondaryOk errors when clients route reads to secondaries without the correct read preference, and prevent stale topology failures after failover.

MongoDB oplog window collapse: secondaries falling off and forced full resync

Diagnose and fix MongoDB oplog window collapse when write surges shrink the replication window and force secondaries into RECOVERING state.

MongoDB oplog window too small: sizing the oplog for your write volume

Size and monitor the MongoDB oplog window to prevent secondaries from falling off and forcing full resyncs. Production rules, resize commands, and thresholds.

MongoDB rollback after failover: silent data loss and the rollback directory

Understand MongoDB rollback after failover, recover data from the rollback directory, and prevent silent data loss with majority write concern.

MongoDB too stale to catch up: secondary stuck in RECOVERING and how to resync

Diagnose and recover a MongoDB secondary that has fallen past the primary's oplog window, is stuck in RECOVERING, and requires a full initial sync.

MongoDB WiredTiger cache pressure cascade: eviction stalls and latency spikes

Diagnose and break the MongoDB WiredTiger cache pressure cascade before eviction stalls and latency spikes bring down your replica set.

MySQL Operations Guides 6 pages

Open the MySQL Operations Guides hub

MySQL Binary Log Disk Full: How To Fix It

Diagnose and fix MySQL binary log disk exhaustion caused by missing expiry, lagging replicas, or oversized transactions. Safe manual purge procedures and prevention.

MySQL Got error 28 from storage engine / No space left on device — recovery

Recover from MySQL errno 28 disk-full conditions by identifying whether binlogs, redo logs, tmpdir, or relay logs consumed the space, and apply the right fix without causing data loss.

MySQL GTID errant transactions: detecting replication divergence

Detect true GTID replication divergence versus ordinary lag, identify errant transactions, and decide between empty-transaction injection or a full rebuild.

MySQL monitoring checklist: the signals every production instance needs

A tiered operational reference of the core MySQL signals every production instance needs, from survival metrics to expert-level lock diagnostics.

MySQL Replica_IO_Running / Replica_SQL_Running not Yes: replication stopped

Diagnose why MySQL replication I/O or SQL threads stop, read SHOW REPLICA STATUS errors, and recover safely.

MySQL replication lag spiral: Seconds_Behind_Source growing without bound

Diagnose and stop a MySQL replication lag death spiral when the replica cannot keep up, relay logs grow, and Seconds_Behind_Source increases monotonically.

PostgreSQL Operations Guides 11 pages

Open the PostgreSQL Operations Guides hub

PostgreSQL autovacuum blocked by long-running transaction: detection and fix

Detect when long-running transactions, replication slots, or standby feedback pin the PostgreSQL xmin horizon and block autovacuum from reclaiming dead tuples.

PostgreSQL backup strategy: pg_dump, pg_basebackup, and pgBackRest compared

Compare logical and physical PostgreSQL backup approaches, pg_dump versus pg_basebackup, pgBackRest status, and what to use in 2026.

PostgreSQL dead tuples piling up: why autovacuum can't keep up

Diagnose why PostgreSQL dead tuple counts grow faster than autovacuum can reclaim them, and fix the root cause before bloat triggers an outage.

PostgreSQL disk full: emergency recovery and root cause analysis

Emergency runbook for PostgreSQL disk-full incidents. Diagnose WAL accumulation, replication slots, temp files, and table bloat without guessing.

PostgreSQL frozen XID monitoring: catching wraparound 6 months early

Monitor datfrozenxid, relfrozenxid, and multixact age with tiered thresholds to catch PostgreSQL transaction ID wraparound while you still have months of runway.

PostgreSQL logical replication failures: conflicts, schema drift, and recovery

Diagnose and resolve PostgreSQL logical replication failures including subscriber conflicts, schema drift gaps, replica identity misconfiguration, and recovery procedures.

PostgreSQL major version upgrade: pg_upgrade, logical replication, and rollback plans

Choose between pg_upgrade and logical replication for PostgreSQL major version upgrades, with rollback strategies and post-upgrade verification.

PostgreSQL Monitoring Checklist: The Signals Every Production Database Needs

A staged monitoring checklist that maps the essential PostgreSQL signals, metrics, and thresholds to operational maturity levels, from basic health to predictive operations.

PostgreSQL pg_wal directory full: causes and emergency recovery

Diagnose unbounded WAL growth in PostgreSQL and recover safely when pg_wal fills the disk, including archive_command failures, replication slot retention, and emergency cleanup.

PostgreSQL replication lag: detection, diagnosis, and fixes

Detect, diagnose, and fix PostgreSQL streaming replication lag before it turns a routine failover into a data-loss event.

PostgreSQL replication slot bloat: when a stale slot fills the disk

Detect when a stale or orphaned PostgreSQL replication slot retains WAL and fills disk, diagnose the root cause, and recover safely.

RabbitMQ Operations Guides 2 pages

Open the RabbitMQ Operations Guides hub

RabbitMQ messages lost on restart: durable queues, persistent messages, and transient traps

Why RabbitMQ messages disappear after a broker restart: the durable queue plus persistent message requirement, transient traps, and the classic queue fsync gap.

RabbitMQ node is quorum critical: checking before a rolling restart

How to use rabbitmq-diagnostics check_if_node_is_quorum_critical and the /api/health/checks/node-is-quorum-critical endpoint to gate rolling restarts so you never take a quorum queue below its online majority.

Redis Operations Guides 9 pages

Open the Redis Operations Guides hub

Redis cluster_slots_pfail > 0: impending node failure in a cluster

Investigate Redis Cluster PFAIL before it escalates to FAIL. Learn the gossip mechanism, quorum rules, zombie states, and how to distinguish transient pauses from real node failure.

Redis FLUSHALL ran in production: detection, prevention, and recovery

Detect, recover from, and prevent accidental or malicious FLUSHALL and FLUSHDB execution in production Redis deployments.

Redis fork/COW memory storm: why persistence doubles RSS and OOM-kills the box

Why Redis background saves trigger copy-on-write memory spikes that double RSS and OOM-kill containers, and how to diagnose, fix, and prevent them.

Redis latest_fork_usec too high: THP, NUMA, and fork latency

Diagnose and fix elevated Redis fork latency caused by Transparent Huge Pages, NUMA misconfiguration, and memory overcommit issues.

Redis MASTERDOWN / master_link_status:down: replication link broken

Diagnose and fix a Redis replica when master_link_status flips to down, the replication link breaks, and stale reads are served.

Redis READONLY You can't write against a read only replica - causes and fixes

Diagnose and fix the Redis READONLY error when applications write to a replica, including routing bugs, stale topology after failover, and runtime misconfigurations.

Redis replication backlog overflow: full-resync storms and the 1MB default

Diagnose and break replication backlog overflow loops in Redis, where a 1MB default triggers full resync storms that fork the primary and cascade latency to every replica.

Redis replication lag: detection, diagnosis, and fixes

Troubleshoot Redis replication lag by measuring byte-level offsets, identifying resource bottlenecks on replicas, and preventing full resync cascades caused by undersized backlogs.

Redis sync_full incrementing: diagnosing full resync events

Diagnose why Redis sync_full counters are climbing, identify replication backlog exhaustion, and stop full resync storms before they overwhelm the primary.

Apache ZooKeeper Operations Guides 2 pages

Open the Apache ZooKeeper Operations Guides hub

ZooKeeper data tree digest mismatch: detecting corruption before it spreads

What zk_digest_mismatches_count means, the four causes to investigate first, and how to triage single-node corruption from a ZAB replication bug.

ZooKeeper quorum loss: no leader elected and every write is failing

Recover from ZooKeeper quorum loss: confirm no leader is elected, identify the cause, and restore enough voting members to form quorum without losing data.

Academy

Elasticsearch Yellow Cluster: Unassigned Shards Fix

A guide to resolving the common Elasticsearch yellow state by investigating shard allocation- understanding master election- and preventing data divergence

Redis Sentinel Failover & Split-Brain Recovery Guide

Learn to decode Redis Sentinel logs to identify network partitions and prevent inconsistent cluster states during failover events

A Comprehensive Guide To Database Performance Optimization

Unlock peak database efficiency and reliability with proven strategies for performance tuning and optimization- Say goodbye to bottlenecks and hello to speed.