$ guides / rabbitmq / rabbitmq-message-ttl-expiry ▌

Operations Guides

RabbitMQ message TTL expiry: messages vanishing before consumers read them

A queue shows a stable, modest depth. Publish rate is healthy. Consumers are connected. Then a downstream team reports missing data, and you discover the queue was never keeping up. Messages were expiring faster than consumers could read them, and RabbitMQ silently discarded every one.

This is the signature failure of message TTL without a dead-letter exchange. A queue-level x-message-ttl argument or a per-message expiration property tells RabbitMQ to discard messages that sit too long. Unless a dead-letter exchange (DLX) is configured, expired messages are dropped with no error, no return to the publisher, and no counter that obviously screams “data loss.” Expiry also works against your monitoring: queue depth stays flat because the backlog is being deleted, and head message age resets every time the head message expires.

This guide covers how to tell whether TTL is doing its intended job or hiding a consumer outage, how to diagnose the discrepancy, and how to stop losing the evidence.

What this means

RabbitMQ supports two TTL mechanisms:

Per-queue TTL: the x-message-ttl queue argument (usually set via a policy). Every message in the queue expires after the configured milliseconds. Because all messages share the same TTL, expired messages are always at the head of the queue and are discarded promptly.
Per-message TTL: the expiration property set by the publisher on each message. Each message carries its own lifetime. When both are set, the lower value wins.

The per-message variant has a behavior that surprises operators: expired messages are only actually discarded when they reach the head of the queue. A message whose TTL has already elapsed can sit behind non-expired messages. It will not be delivered to consumers, but it still occupies memory and is still counted in queue statistics until it reaches the head. Queue depth can therefore overstate deliverable work while understating real consumer lag.

There is also a natural race between expiration and delivery: a message can expire between the broker deciding to deliver it and the consumer receiving it. Consumers never receive expired messages, but the boundary is not a precise contract.

The result is a queue that looks healthy while it quietly sheds work. If that shedding matches a deliberate design decision (a time-bounded cache invalidation queue, a presence stream where stale events are worthless), fine. If the TTL was set years ago and the consumer fleet has since degraded, TTL is now a silent data-loss valve masking a consumer outage.

flowchart LR
  P[Publishers] --> Q[Queue with TTL]
  Q --> C[Consumers]
  Q -->|TTL expired, no DLX| X[Silently dropped]
  Q -->|TTL expired, DLX set| D[Dead-letter exchange]
  D --> DLQ[Dead-letter queue: evidence preserved]
  C -.->|if consumers lag| LAG[Backlog grows]
  LAG -.->|expiry deletes head| MASK[Depth looks stable: backlog masked]

Common causes

Cause	What it looks like	First thing to check
Consumer outage hidden by TTL	Depth stable or low, publish rate normal, downstream reports gaps	Ack rate vs publish rate; consumer count and application health
TTL tuned for an old traffic profile	Expiry rate climbs gradually over weeks as traffic grows	Policy value (`rabbitmqctl list_policies`) vs current publish rate and consumer capacity
Per-message TTL head-of-queue distortion	Depth looks nonzero but consumers starve; expired messages pile behind live ones	Per-message `expiration` usage; classic vs quorum queue type
TTL set to 0 by mistake or intent	Messages vanish instantly unless a consumer is ready at publish time	Queue arguments for `x-message-ttl: 0`
DLX configured but exchange missing	Messages still silently dropped despite DLX arguments	The dead-letter exchange actually exists and has bindings
Slow consumers under a tight TTL	Depth flat, but `consumer_utilisation` low and ack rate below publish rate	Per-queue consumer utilisation and ack rate
Deliberate TTL doing its job	DLX queue depth low and stable, downstream has no gaps	Confirm this is the documented design, not drift

Quick checks

All of these are read-only.

# Find policies and queue arguments that set TTL
rabbitmqctl list_policies
rabbitmqctl list_queues name arguments

# Per-queue depth, unacked, and consumer count
rabbitmqctl list_queues name messages_ready messages_unacknowledged consumers

# Cluster-wide rates: publish vs deliver vs ack
curl -s -u guest:guest http://localhost:15672/api/overview | jq '.message_stats | {publish, deliver_get, ack, redeliver}'

# Per-queue detail for a suspect queue: depth, head age, consumer utilisation
curl -s -u guest:guest 'http://localhost:15672/api/queues/%2f/my_queue' | \
  jq '{messages_ready, messages_unacknowledged, consumers, consumer_utilisation, head_message_timestamp, arguments}'

# Check whether a dead-letter queue exists and is accumulating
rabbitmqctl list_queues name messages | grep -i dlx

The management API examples use the default guest user, which only accepts connections from localhost. Substitute real credentials for remote checks.

How to diagnose it

Establish the accounting discrepancy. The core evidence for silent expiry is that throughput does not add up: publish rate is healthy, deliver/ack rate is lower than publish, and yet queue depth is not growing. The missing messages are going somewhere. With no DLX, that somewhere is nowhere.
Find the TTL configuration. Run rabbitmqctl list_policies and check the queue’s arguments. Look for message-ttl in a policy or x-message-ttl in the queue declaration. Also check whether publishers set the expiration property per message, since per-message TTL will not appear in broker configuration at all. You may need to inspect publisher code or fetch a message from the queue to confirm.
Compare the TTL against reality. If consumers are healthy, head-of-queue wait time should be far below the TTL and expiry should be rare. If wait time approaches or exceeds the TTL, expiry is load-bearing: it is the only thing keeping depth flat.
Check head message age, with skepticism. head_message_timestamp (3.8+) is the best latency signal, but it misleads in exactly this scenario: every time the head message expires, the head resets to a newer message, so age can look fine while the queue discards data. The field also depends on publishers setting the timestamp property.
Rule in or rule out a consumer problem. Check consumers, consumer_utilisation, ack rate, and the consumer application’s own health. The classic finding: consumers connected, utilisation low, ack rate below publish rate, depth flat because TTL is absorbing the difference. That is a consumer outage wearing a TTL disguise.
Check queue type if per-message TTL is in play. On quorum queues, expired messages are dead-lettered when they reach the head. On classic queues, expiry also happens when the queue is notified of a policy change. Classic priority queues have an extra wrinkle: a high-priority short-TTL message behind low-priority long-TTL messages is not expired until the messages ahead of it are consumed or expire. If behavior differs from what you expect, queue type is the first thing to verify.
Verify the DLX path actually exists. If the queue has x-dead-letter-exchange set, confirm that exchange exists and has a binding to a real queue. A DLX argument pointing at a nonexistent exchange silently drops dead-lettered messages, which recreates the original problem with extra steps.

Metrics and signals to monitor

Signal	Why it matters	Warning sign
Publish vs deliver_get vs ack rate	The accounting identity that exposes silent loss	Publish sustained above ack while depth stays flat
messages_ready trend	With expiry, flat depth no longer means “keeping up”	Flat depth alongside a publish/ack gap
head_message_timestamp	Latency floor, but resets on expiry	Age hovering just below the TTL value, repeatedly
consumer_utilisation	Shows whether attached consumers can actually keep up	Below 0.5 sustained with messages ready
Dead-letter queue depth and rate	The only durable record of expiry (if DLX configured)	Any sustained growth; spikes correlate with consumer degradation
redeliver rate	Distinguishes expiry from consumer rejection loops	Elevated redeliver points at poison messages, not TTL
Consumer count	Zero consumers on a TTL queue means every message is on a countdown	Consumers drop to 0 and depth still decays

Fixes

Route expired messages to a DLX

This is the single most important fix, even when TTL is intentional. Set x-dead-letter-exchange (via policy, so it applies without redeclaring queues) and bind a dead-letter queue. Expired messages then land there with an x-death header recording the reason (expired), the original queue, and a count. You get an observable expiry rate instead of a silent hole. When a message is dead-lettered, its per-message TTL is removed so it does not immediately expire again in the target queues; the original value is preserved in the x-death header as original-expiration.

Tradeoff: the dead-letter queue needs its own ownership. An unmonitored DLX queue with no consumer grows until it becomes its own incident. Alert on its depth and arrival rate.

Fix the consumer problem TTL was hiding

If diagnosis shows consumers cannot drain within the TTL, the TTL is not the bug; it is the symptom absorber. Scale consumers, fix the downstream dependency slowing them, or reduce prefetch so work spreads across the fleet. Do this before touching the TTL value, because raising the TTL without fixing consumption just converts silent loss into a memory wall later.

Adjust or remove the TTL deliberately

If expiry is no longer an acceptable semantic for this queue, remove the message-ttl policy or raise it well above worst-case consumer lag. Do this with capacity awareness: a queue that previously self-trimmed via TTL will now accumulate, so confirm memory headroom (mem_used / mem_limit) and disk headroom before removing the safety valve.

Use TTL=0 only with eyes open

x-message-ttl: 0 expires messages on arrival unless they can be delivered to a consumer immediately. It is the supported stand-in for the removed immediate flag, and it produces no returns to the publisher. That is fine for presence-style traffic, catastrophic for anything else. If you find it on a queue that carries real work, treat it as a misconfiguration until proven otherwise.

Prefer quorum queues for TTL workloads

Quorum queues handle per-message TTL expiry and dead-lettering more predictably than classic queues (head-based expiry, at-least-once dead-lettering available since 3.10), and classic mirrored queues are removed entirely in 4.x. One caution from the field: a quorum queue with TTL and no consumers can grow Raft segment files unboundedly because nothing is ever acknowledged, so compaction never advances. If you use a TTL queue as a delay buffer with no consumers, watch its disk usage specifically. To cap segment growth, current RabbitMQ versions expose quorum_queue.segment_max_size_bytes (default 64 MB); the older per-entry knob is no longer recommended.

Prevention

Policy review: TTL values should have an owner and a justification. Any message-ttl policy without a comment, ticket, or design note is drift waiting to happen.
DLX everywhere TTL exists: treat TTL without DLX as an incomplete configuration. The DLX is not optional decoration; it is the audit trail.
Alert on the accounting gap: publish rate sustained above ack rate with flat depth is a first-class alert condition on TTL queues, because depth alone will never fire.
Monitor the dead-letter queue: depth, arrival rate, and the x-death reason mix. expired dominating means consumers are too slow; rejected means poison messages; maxlen means capacity problems.
Track consumer lag against the TTL: the ratio of head wait time to TTL is your leading indicator. Alert before wait time reaches the TTL, not after messages start vanishing.
Reconsider the TTL+DLX retry pattern: RabbitMQ 4.3.0 (April 2026) added native delayed retry for quorum queues, intended to replace the old pattern of using TTL plus dead-lettering to implement retry delays. The classic TTL+DLX cycle also carried at-most-once dead-lettering loss risk on classic queues.

How Netdata helps

Rate correlation in one view: publish, deliver_get, and ack rates charted together make the accounting gap (publish above ack, flat depth) visible in seconds instead of requiring manual API polling.
Per-queue depth and unacked tracking: separates ready buildup from in-flight buildup, so you can tell a consumer stall apart from expiry-driven trimming.
Consumer count and utilisation signals: connecting “consumers attached” to “consumers effective” is what distinguishes a real outage from healthy TTL behavior.
Head message age charting: exposes the telltale sawtooth of age repeatedly resetting near the TTL, which is the visual signature of expiry masking lag.
Dead-letter queue alerting: once a DLX is configured, Netdata can alert on its growth rate, turning silent expiry into a pageable, measurable signal.