Message Queue Monitoring

Message queue monitoring tracks queue depth, throughput, consumers and broker health for systems such as ActiveMQ, Apache Pulsar and RabbitMQ. This hub groups Netdata's ranked lists of monitoring tools for those brokers.

Search Message Queue Monitoring pages

Guides

Guides overview 2 pages

Kafka Operations Guides

Practical guides for running, troubleshooting, and monitoring Apache Kafka in production.

RabbitMQ Operations Guides

Practical guides for running, troubleshooting, and monitoring RabbitMQ in production.

Kafka Operations Guides 24 pages

Open the Kafka Operations Guides hub

Kafka ActiveControllerCount not equal to 1: no controller or split brain

Diagnose and recover when the cluster-wide sum of Kafka ActiveControllerCount deviates from 1, indicating no active controller or split brain.

Kafka Broker May Not Be Available: How To Fix It

Diagnose why Kafka clients fail with 'Broker may not be available' even when the bootstrap server is reachable. Covers advertised.listeners misconfiguration, TLS and SASL handshake failures, firewall blocks, and genuine broker outages.

Kafka CommitFailedException: rebalanced-out consumers and poll loop timeouts

Operational guide to diagnosing CommitFailedException caused by consumers evicted after exceeding max.poll.interval.ms during slow batch processing, and stopping rebalance storms.

Kafka consumer group lag growing: detection, lag-as-time, and root causes

Diagnose growing Kafka consumer group lag, convert offsets to time, and identify root causes from slow consumers to broker-side page cache and disk bottlenecks.

Kafka consumer group rebalancing too often: heartbeats, session timeout, and assignors

Diagnose and fix Kafka consumer groups that rebalance too frequently. Covers heartbeat timeouts, session expiry, max.poll.interval.ms, assignor behavior, and static membership.

Kafka consumer rebalance storm: stuck in PreparingRebalance and max.poll.interval.ms

Diagnose and fix Kafka consumer groups that oscillate between Stable and PreparingRebalance, driven by max.poll.interval.ms timeouts and client-side processing delays.

Kafka controller event queue backing up: overwhelmed controller and stalled metadata

Diagnose and recover when the Kafka controller's event queue backs up, stalling leader elections and metadata propagation.

Kafka disk I/O latency high: await, LocalTimeMs, and the slow-disk broker

Diagnose elevated disk I/O latency on Kafka brokers using iostat await and LocalTimeMs. Distinguish disk degradation from page cache misses and transient load.

Kafka FailedProduceRequestsPerSec rising: the single best 'producers are hurting' signal

Diagnose rising Kafka FailedProduceRequestsPerSec, correlate with ISR and replication health, and stop producer-visible failures before they cascade.

Kafka ISR shrinking: IsrShrinksPerSec, flapping, and the cascade to offline

Diagnose ISR shrink storms, flapping replicas, and the cascade to offline partitions in Kafka production clusters.

Kafka LEADER_NOT_AVAILABLE: causes during elections, restarts, and topic creation

Diagnose transient and persistent Kafka LEADER_NOT_AVAILABLE errors by correlating leader elections, controller health, ISR state, and offline partitions.

Kafka log compaction falling behind: the dead cleaner thread and unbounded disk growth

Diagnose and fix silent Kafka log cleaner thread crashes that cause compacted topics like __consumer_offsets to grow without bound.

Kafka Log directory failed / OfflineLogDirectoryCount > 0: disk errors and JBOD recovery

Troubleshoot Kafka brokers with offline log directories due to disk I/O errors, JBOD failures, and recover partition availability without data loss.

Kafka monitoring checklist: the signals every production cluster needs

A production monitoring checklist for Apache Kafka organized by maturity level: survival, operational, mature, and expert signals every SRE should watch.

Kafka NetworkProcessorAvgIdlePercent low: network thread saturation and TLS overhead

Diagnose and fix low NetworkProcessorAvgIdlePercent in Kafka, including network thread saturation from TLS handshakes, connection storms, and large fetch responses.

Kafka NOT_LEADER_FOR_PARTITION: stale metadata, controller lag, and client retries

Diagnose persistent NOT_LEADER_FOR_PARTITION errors by distinguishing transient leader elections from controller event queue backup and broker metadata desync.

Kafka NotEnoughReplicasException: acks=all writes rejected below min.insync.replicas

Diagnose and fix Kafka NotEnoughReplicasException when acks=all writes fail because the ISR shrank below min.insync.replicas.

Kafka OfflinePartitionsCount > 0: partitions with no leader and how to recover

Diagnose and recover Kafka partitions with no active leader when OfflinePartitionsCount is nonzero.

Kafka OffsetOutOfRangeException: when retention deletes data before the consumer reads it

Fix and prevent Kafka OffsetOutOfRangeException when consumer lag outruns log retention, causing data loss or forced offset resets.

Kafka RecordTooLargeException / MESSAGE_TOO_LARGE: message size limits across the path

Fix Kafka message-size rejections by aligning producer, broker, topic, replica, and consumer limits end to end.

Kafka replica MaxLag growing: slow followers and replica fetcher health

Diagnose and fix growing Kafka replica fetcher lag before it triggers ISR shrinks and write rejections.

Kafka REQUEST_TIMED_OUT: produce requests that expire before replication completes

Diagnose and fix Kafka REQUEST_TIMED_OUT errors when acks=all produce requests expire waiting for replication, request queue backup, or slow disk.

Kafka UncleanLeaderElectionsPerSec > 0: confirmed silent data loss

What to do when Kafka's UncleanLeaderElectionsPerSec metric rises above zero, confirming silent data loss, and how to recover without deepening the damage.

Kafka UnderReplicatedPartitions > 0: the most important metric and how to clear it

Troubleshoot and clear Kafka under-replicated partitions by identifying lagging followers, correlating disk, network, and GC signals, and applying the right fix without causing further ISR shrink.

RabbitMQ Operations Guides 20 pages

Open the RabbitMQ Operations Guides hub

RabbitMQ ACCESS_REFUSED - Login was refused: authentication failures and credential rotation

Diagnose RabbitMQ 'ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN' errors: stale credentials after rotation, disabled users, guest-from-remote, LDAP backend failures, and brute-force floods, plus how to stop the reconnect churn they cause.

RabbitMQ connection blocked / blocking: publishers frozen by a resource alarm

Why RabbitMQ connections enter the blocking or blocked state, how to tell it apart from flow control, and how to unblock publishers frozen by a memory or disk resource alarm.

RabbitMQ connection in flow state: credit-based backpressure explained

What RabbitMQ's per-connection flow state actually means, how credit-based backpressure works inside the broker, and when sustained flow is a real write-path bottleneck.

RabbitMQ connection storm: reconnect loops, FD pressure, and CPU spent on handshakes

Diagnose and stop a RabbitMQ connection storm: reconnect loops driving churn, file descriptor and socket pressure, and Erlang scheduler CPU burned on connection handshakes.

RabbitMQ consumer timeout: delivery acknowledgement timed out and the channel is closed

Why RabbitMQ closes channels with 'delivery acknowledgement timed out', how the consumer_timeout mechanism works, and how to diagnose and fix slow or stuck consumers.

RabbitMQ consumers connected but not acknowledging: the consumer black hole

RabbitMQ shows consumers attached but ack rate is zero and messages_ready is rising. How to diagnose stuck consumers, zombie connections, and downstream stalls before the memory alarm fires.

RabbitMQ disk free limit alarm: free disk space insufficient and publishing halted

Diagnose and recover from the RabbitMQ disk_free_alarm that blocks all publishers cluster-wide when free disk drops below disk_free_limit.

RabbitMQ disk_free_limit at the default 50MB: the production footgun

Why RabbitMQ's default 50MB disk_free_limit halts publishing cluster-wide on any modern server, and how to set a safe absolute or relative limit with a sane alerting floor.

RabbitMQ file descriptor exhaustion: fd_used near fd_total and refused connections

Diagnose and fix RabbitMQ file descriptor exhaustion when fd_used approaches fd_total, new connections are refused, and queue file operations start failing.

RabbitMQ flow control vs resource alarms: the two throttling mechanisms operators confuse

RabbitMQ has two independent throttling mechanisms: per-connection credit-based flow control (state: flow) and cluster-wide resource alarms (state: blocked). Learn to tell them apart and alert on each correctly.

RabbitMQ memory resource limit alarm: publishers blocked across the whole cluster

What the RabbitMQ memory resource limit alarm means, why one node blocks publishers cluster-wide, and how to find and fix the cause without restarting the broker.

RabbitMQ memory usage climbing toward the watermark: mem_used vs mem_limit

How to read RabbitMQ's mem_used and mem_limit, why the ratio is the leading gauge before the memory alarm, and what to do when it climbs past 0.7.

RabbitMQ messages lost on restart: durable queues, persistent messages, and transient traps

Why RabbitMQ messages disappear after a broker restart: the durable queue plus persistent message requirement, transient traps, and the classic queue fsync gap.

RabbitMQ monitoring checklist: the signals every production broker needs

A four-level RabbitMQ monitoring checklist for production brokers, from survival signals like mem_alarm and disk_free_alarm to expert signals like head message age and consumer utilisation.

RabbitMQ network partition detected: split-brain in cluster_status

What the Network Partitions section of rabbitmqctl cluster_status means, how to confirm a real split versus a false positive, and how to respond without making data divergence worse.

RabbitMQ node down: telling a dead broker apart from a partitioned one

How to distinguish a genuinely dead RabbitMQ node from a network-partitioned one, using the running field, partitions array, cluster link traffic, and the quorum-critical check.

RabbitMQ node is quorum critical: checking before a rolling restart

How to use rabbitmq-diagnostics check_if_node_is_quorum_critical and the /api/health/checks/node-is-quorum-critical endpoint to gate rolling restarts so you never take a quorum queue below its online majority.

RabbitMQ NOT_FOUND - no queue or exchange: publishing to something that does not exist

Why RabbitMQ closes your channel with 404 NOT_FOUND when you publish to or consume from an exchange or queue that does not exist, how to find what is missing, and how to stop it recurring after deploys.

RabbitMQ PRECONDITION_FAILED - inequivalent arg: redeclaring a queue with different properties

Why RabbitMQ closes the channel with 406 PRECONDITION_FAILED when a client redeclares a queue or exchange with different arguments, and how to reconcile it safely.

RabbitMQ RESOURCE_LOCKED - cannot obtain exclusive access to queue

Diagnose and fix RabbitMQ channel error 405 RESOURCE_LOCKED when a second connection tries to use an exclusive queue still owned by another connection.

Academy

Apache Kafka Consumer Lag: Troubleshooting & Fixes

A deep dive into the causes of consumer lag- how to monitor it- and how to tune your Kafka consumers for optimal performance

What Is Apache Kafka Used For Everything You Need To Know

Unpacking the power of Apache Kafka for modern data pipelines - from real-time analytics to robust messaging systems.

ActiveMQ Monitoring

Learn everything about monitoring & troubleshooting ActiveMQ, what metrics are important to monitor and why, and how to monitor ActiveMQ with Netdata.

Apache Pulsar Monitoring

Learn everything about monitoring & troubleshooting Apache Pulsar, what metrics are important to monitor and why, and how to monitor Apache Pulsar with Netdata.

Beanstalk Monitoring

Learn everything about monitoring & troubleshooting Beanstalk, what metrics are important to monitor and why, and how to monitor Beanstalk with Netdata.

Celery Monitoring

Learn everything about monitoring & troubleshooting Celery, what metrics are important to monitor and why, and how to monitor Celery with Netdata.

Gearman Monitoring

Learn everything about monitoring & troubleshooting Gearman, what metrics are important to monitor and why, and how to monitor Gearman with Netdata.

IBM MQ Monitoring

Learn everything about monitoring & troubleshooting IBM MQ, what metrics are important to monitor and why, and how to monitor IBM MQ with Netdata.

Kafka Consumer Lag Monitoring

Learn everything about monitoring & troubleshooting Kafka Consumer Lag, what metrics are important to monitor and why, and how to monitor Kafka Consumer Lag with Netdata.

Kafka Monitoring

Learn everything about monitoring & troubleshooting Kafka, what metrics are important to monitor and why, and how to monitor Kafka with Netdata.

Mosquitto Monitoring

Learn everything about monitoring & troubleshooting Mosquitto, what metrics are important to monitor and why, and how to monitor Mosquitto with Netdata.

MQTT Blackbox Monitoring

Learn everything about monitoring & troubleshooting MQTT Blackbox, what metrics are important to monitor and why, and how to monitor MQTT Blackbox with Netdata.

NATS Monitoring

Learn everything about monitoring & troubleshooting NATS, what metrics are important to monitor and why, and how to monitor NATS with Netdata.

RabbitMQ Monitoring

Learn everything about monitoring & troubleshooting RabbitMQ, what metrics are important to monitor and why, and how to monitor RabbitMQ with Netdata.

Redis Queue Monitoring

Learn everything about monitoring & troubleshooting Redis Queue, what metrics are important to monitor and why, and how to monitor Redis Queue with Netdata.

VerneMQ Monitoring

Learn everything about monitoring & troubleshooting VerneMQ, what metrics are important to monitor and why, and how to monitor VerneMQ with Netdata.

Resources

7 Best Apache Pulsar Monitoring Tools Ranked (2026)

Compare the 7 best Apache Pulsar monitoring tools for broker health, BookKeeper storage, and consumer backlog, ranked on depth, setup effort, and cost.

Best ActiveMQ Monitoring Tools: 10 Ranked for 2026

Compare the 10 best ActiveMQ monitoring tools of 2026: queue depth coverage, JMX setup effort, alerting depth, and pricing predictability, ranked for engineers.

Best Kafka Monitoring Tools: 10 Ranked for 2026

We ranked 10 Kafka monitoring tools on consumer lag depth, collection resolution, alerting, and cost. See where Netdata, Prometheus, and Datadog land.

Best NATS Monitoring Tools: 9 Ranked for 2026

We ranked the 9 best NATS monitoring tools for 2026, grading JetStream depth, collection resolution, alerting, and cost for production messaging fleets.

Best RabbitMQ Monitoring Tools (2026): 10 Ranked & Reviewed

Compare the 10 best RabbitMQ monitoring tools for 2026: queue depth, message rates, and broker health tracked by Netdata, Prometheus, Datadog, and more.