High Availability

Product

Netdata Parents For Intelligent Observability

Monitor millions of metrics per second with low overhead, distributed ML & resilient active-active clustering. Smarter observability at scale. Book a demo!

Zero-Downtime Monitoring For Real-Time Deployments

Ensure operational continuity with autonomous agents, automatic failover, and ML-powered anomaly detection for real-time deployments. Read more now!

Guides

Ceph Operations Guides 1 page

Open the Ceph Operations Guides hub

Ceph FS_DEGRADED: standby MDS failed to take over a rank

Diagnose CephFS FS_DEGRADED when a standby MDS does not promote to a failed rank, and recover before the cluster escalates to MDS_ALL_DOWN.

CockroachDB Operations Guides 5 pages

Open the CockroachDB Operations Guides hub

CockroachDB clock skew cascade: how shared NTP drift causes quorum loss

Diagnose and resolve CockroachDB clock skew cascades where shared NTP drift causes multiple node self-terminations and quorum loss.

CockroachDB connection storm after failover: reconnect stampedes and surviving-node overload

Diagnose and mitigate reconnect stampedes that overload surviving CockroachDB nodes after failover, with metrics, quick checks, and client pool tuning.

CockroachDB node liveness failure: heartbeats, lease redistribution, and flapping

Diagnose and fix CockroachDB node liveness failures, from disk stalls and GC pauses to network partitions and flapping availability.

CockroachDB replica unavailable: lost quorum and stuck Raft groups

Diagnose and resolve CockroachDB replica unavailable errors, lost Raft quorum, and stuck Raft groups causing range unavailability.

CockroachDB under-replicated ranges: ranges_underreplicated and the healing margin

Diagnosing CockroachDB under-replicated ranges, understanding the healing margin, and configuring alerting that separates expected recovery from dangerous fault tolerance loss.

Elasticsearch Operations Guides 3 pages

Open the Elasticsearch Operations Guides hub

Elasticsearch master instability: frequent elections and metadata overload

Diagnose and fix Elasticsearch master node flapping caused by cluster state metadata overload, GC pauses, and election timeouts.

Elasticsearch master_not_discovered_exception: no elected master and stalled writes

Diagnose and resolve Elasticsearch master_not_discovered_exception when no master is elected and writes stall due to quorum loss, GC pauses, network partitions, or cluster state overload.

Elasticsearch node left the cluster: fault detection, reallocation, and recovery

Diagnose and recover when an Elasticsearch node departs unexpectedly, including fault detection mechanics, delayed reallocation, and voting configuration cleanup.

HAProxy Operations Guides 1 page

Open the HAProxy Operations Guides hub

HAProxy backend losing servers: active server count and cascade risk

How to detect and respond when HAProxy backends lose servers: the act/bck signals, why BACKEND status stays UP, and how to judge cascade risk before the last server fails.

Kafka Operations Guides 3 pages

Open the Kafka Operations Guides hub

Kafka ISR shrinking: IsrShrinksPerSec, flapping, and the cascade to offline

Diagnose ISR shrink storms, flapping replicas, and the cascade to offline partitions in Kafka production clusters.

Kafka min.insync.replicas and acks: configuring durability you actually have

Why acks=all without min.insync.replicas only guarantees leader durability, and how to configure Kafka replication for real fault tolerance without blocking writes.

Kafka NotEnoughReplicasException: acks=all writes rejected below min.insync.replicas

Diagnose and fix Kafka NotEnoughReplicasException when acks=all writes fail because the ISR shrank below min.insync.replicas.

Kubernetes Operations Guides 1 page

Open the Kubernetes Operations Guides hub

Kubernetes Controller-Manager Leader Election Failures

Diagnose lease renewal failures, leaderless windows, and controller reconciliation stalls in the kube-controller-manager.

Microsoft SQL Server Operations Guides 4 pages

Open the Microsoft SQL Server Operations Guides hub

SQL Server AG send and redo queues growing: replication lag and failover RTO

Diagnose growing log_send_queue_size and redo_queue_size in Always On Availability Groups, estimate failover RTO, and resolve replication lag.

SQL Server Availability Group not synchronizing: NOT_HEALTHY replicas and failover risk

Diagnose and resolve SQL Server Availability Group replicas stuck in NOT_HEALTHY or DISCONNECTED states, and understand when failover capability is actually compromised.

SQL Server database in SUSPECT or RECOVERY_PENDING: an offline database and how to recover it

Diagnose and recover a SQL Server database stuck in SUSPECT, RECOVERY_PENDING, or EMERGENCY without jumping straight to data-loss repair.

SQL Server instance down: no response on port 1433 and where to look first

Triage a SQL Server instance that fails connectivity probes on port 1433 by separating true service outages from worker exhaustion, listener misconfiguration, and database-scoped failures.

MongoDB Operations Guides 4 pages

Open the MongoDB Operations Guides hub

MongoDB no primary / election storm: repeated elections and write outages

Diagnose and stop MongoDB replica set election storms where primaries repeatedly step down, causing rolling 2-12 second write outages.

MongoDB not master error: writes hitting a non-primary node after failover

Troubleshoot and fix MongoDB NotWritablePrimary errors that occur when applications write to a non-primary node after failover or election.

MongoDB not primary and secondaryOk=false: reading from a secondary and how to fix it

Fix MongoDB NotPrimaryNoSecondaryOk errors when clients route reads to secondaries without the correct read preference, and prevent stale topology failures after failover.

MongoDB rollback after failover: silent data loss and the rollback directory

Understand MongoDB rollback after failover, recover data from the rollback directory, and prevent silent data loss with majority write concern.

MySQL Operations Guides 1 page

Open the MySQL Operations Guides hub

MySQL GTID errant transactions: detecting replication divergence

Detect true GTID replication divergence versus ordinary lag, identify errant transactions, and decide between empty-transaction injection or a full rebuild.

PostgreSQL Operations Guides 1 page

Open the PostgreSQL Operations Guides hub

PostgreSQL replication lag: detection, diagnosis, and fixes

Detect, diagnose, and fix PostgreSQL streaming replication lag before it turns a routine failover into a data-loss event.

RabbitMQ Operations Guides 3 pages

Open the RabbitMQ Operations Guides hub

RabbitMQ network partition detected: split-brain in cluster_status

What the Network Partitions section of rabbitmqctl cluster_status means, how to confirm a real split versus a false positive, and how to respond without making data divergence worse.

RabbitMQ node down: telling a dead broker apart from a partitioned one

How to distinguish a genuinely dead RabbitMQ node from a network-partitioned one, using the running field, partitions array, cluster link traffic, and the quorum-critical check.

RabbitMQ node is quorum critical: checking before a rolling restart

How to use rabbitmq-diagnostics check_if_node_is_quorum_critical and the /api/health/checks/node-is-quorum-critical endpoint to gate rolling restarts so you never take a quorum queue below its online majority.

Redis Operations Guides 3 pages

Open the Redis Operations Guides hub

Redis cluster_slots_pfail > 0: impending node failure in a cluster

Investigate Redis Cluster PFAIL before it escalates to FAIL. Learn the gossip mechanism, quorum rules, zombie states, and how to distinguish transient pauses from real node failure.

Redis CLUSTERDOWN / cluster_state:fail: slot coverage and recovery

Diagnose and recover when Redis Cluster reports cluster_state:fail or clients see CLUSTERDOWN due to missing hash slot coverage.

Redis READONLY You can't write against a read only replica - causes and fixes

Diagnose and fix the Redis READONLY error when applications write to a replica, including routing bugs, stale topology after failover, and runtime misconfigurations.

VMware vSphere Operations Guides 1 page

Open the VMware vSphere Operations Guides hub

vSphere HA 'Insufficient resources to satisfy configured failover level': admission control

Why vSphere HA blocks power-ons and vMotions with the failover level error, how admission control policies work, and how to fix it without losing the restart guarantee.

Apache ZooKeeper Operations Guides 1 page

Open the Apache ZooKeeper Operations Guides hub

ZooKeeper quorum loss: no leader elected and every write is failing

Recover from ZooKeeper quorum loss: confirm no leader is elected, identify the cause, and restore enough voting members to form quorum without losing data.

Academy

Elasticsearch Yellow Cluster: Unassigned Shards Fix

A guide to resolving the common Elasticsearch yellow state by investigating shard allocation- understanding master election- and preventing data divergence

Consul Service Discovery Failures: Causes & Fixes

Unravel common errors in Consul cluster formation- service registration- and DNS to ensure a highly available service mesh and reliable discovery

Redis Sentinel Failover & Split-Brain Recovery Guide

Learn to decode Redis Sentinel logs to identify network partitions and prevent inconsistent cluster states during failover events

Ecommerce Infrastructure: Components & Benefits

The essential technical foundation for building and scaling online stores

What Is Database Clustering? Types & Benefits

Enhance Performance, Scalability & Availability Of Databases

6 + 1 Effective Strategies to Reduce Unplanned Downtime

Practical Approaches for Ensuring Uptime and Business Continuity

How To Achieve High Availability In CI/CD With Observability

A practical guide to making your CI-CD pipeline more reliable and efficient with comprehensive monitoring

Keepalived Monitoring

Learn everything about monitoring & troubleshooting Keepalived, what metrics are important to monitor and why, and how to monitor Keepalived with Netdata.

Patroni Monitoring

Learn everything about monitoring & troubleshooting Patroni, what metrics are important to monitor and why, and how to monitor Patroni with Netdata.

Blog

Netdata Parents (Streaming and Replication)

Enhancing Data Reliability and Accessibility Across Networks

Infinite Scalability: Monitoring Without Limits

How Netdata Scales Without Limits

Server Uptime Monitoring: Core Benefits For High Performance

Ensuring High Availability With Continuous Monitoring

Why is data replication important?

Ensuring Data Availability and Integrity Across Systems