HAProxy compression overhead: comp_byp, CPU cost, and when compression backfires

HTTP compression in HAProxy is a trade: CPU cycles on the proxy in exchange for fewer bytes on the wire. When it works, text responses shrink by 50 to 80 percent and clients on slow links see faster page loads. When it backfires, the proxy burns event-loop CPU compressing content that does not compress, or it sheds compression work under load and you lose the bandwidth savings exactly when traffic is highest.

None of this is logged anywhere. HAProxy exposes four counters, comp_in, comp_out, comp_byp, and comp_rsp, and the operator has to do the arithmetic. A comp_out / comp_in ratio near 1.0 means you are paying the CPU cost and getting nothing. A rising comp_byp under load means HAProxy has started skipping compression to protect itself.

This article assumes you already have compression enabled and are deciding whether to keep it, tune it, or turn it off. If compression is not configured, these counters stay at zero and this article does not apply.

What the compression engine costs you

Compression in HAProxy runs inside the event loop, on the same threads that accept connections, parse HTTP, and do TLS handshakes. Every byte through the compressor is CPU time taken from the same budget as everything else the proxy does. There is no supported option to offload compression to a separate worker or process; compression offload only makes HAProxy strip Accept-Encoding from client requests so backends do not compress instead.

The dominant CPU costs on a terminating proxy, in rough order, are TLS handshakes first, then ACL evaluation, regex matching, compression, Lua scripts, header parsing, stick-table lookups, and logging. Unlike TLS, which you usually cannot avoid, compression is optional, which makes it the first thing to question when the event loop runs out of headroom.

The failure shape is not a crash. Compression pressure shows up as falling Idle_pct: the percentage of time the event loop spends waiting in poll() instead of working. As compression load grows, Idle_pct falls, latency rises smoothly, and below roughly 5 percent idle the proxy degrades hard: connections time out, health checks get flaky, and the whole proxy looks “slow” with no single obvious cause.

flowchart TD
  A[Backend response arrives] --> B{Matches compression rules?}
  B -->|content type or size excluded| C[Forwarded uncompressed]
  B -->|already compressed| C
  B -->|eligible| D{CPU or bandwidth limit hit?}
  D -->|yes: maxcompcpuusage or maxcomprate| E[Bypass: counted in comp_byp]
  D -->|no| F[Compress in event loop]
  F --> G[comp_in and comp_out increment]
  C --> H[Client]
  E --> H
  G --> H

The four compression counters

HAProxy exposes compression activity as four cumulative counters in the stats CSV, on FRONTEND and BACKEND rows. They only carry meaningful values when compression is configured.

Counter	CSV field (0-based)	What it counts
comp_in	51	Bytes fed into the compressor (pre-compression size)
comp_out	52	Bytes emitted by the compressor (post-compression size)
comp_byp	53	Bytes that bypassed compression entirely
comp_rsp	54	Number of responses that were compressed

Collect them from the runtime socket:

# Compression counters per frontend and backend
echo "show stat" | socat unix-connect:/var/run/haproxy.sock stdio | \
  awk -F, '$2 == "FRONTEND" || $2 == "BACKEND" {print $1"/"$2": comp_in="$52" comp_out="$53" comp_byp="$54" comp_rsp="$55}'

The awk offsets are 1-based while the CSV field documentation is 0-based. Verify against the header line (the first line of show stat output starts with # and lists field names) on your version, since field numbering can shift between HAProxy releases.

These are cumulative counters that reset on every reload, like the rest of the CSV stats. Derive rates and ratios from deltas, and treat a counter reset as a reload, not an incident.

Reading the ratios

Three derived numbers carry the operational meaning.

Compression ratio: comp_out / comp_in. For compressible text (HTML, CSS, JavaScript, JSON), expect 0.2 to 0.5, a 50 to 80 percent size reduction. A ratio near 1.0 means the compressor is running but producing almost no savings: the content is already compressed (images, video, PDFs, archives) or too small to benefit. That is pure CPU waste.

Bypass fraction: comp_byp / (comp_in + comp_byp). Some bypass is normal. Responses that already carry a Content-Encoding header, that do not match the configured compression type rules, or that fall below a configured size threshold are counted in comp_byp, and skipping them is correct behavior. The signal is in the trend: if the bypass fraction climbs during traffic peaks while comp_in stays flat or falls, HAProxy is shedding compression work because its CPU or bandwidth limits for compression have been reached. You lose the bandwidth savings at exactly the moment bandwidth matters most.

Bytes compressed per response: comp_in / comp_rsp. If the average compressed response is tiny, you are paying per-response compression overhead (headers, checksums, event-loop time) for payloads where the compressed output can be as large as the input.

When compression backfires

Compressing the incompressible. The most common misconfiguration is a compression type list that is too broad, or no type restriction at all. JPEG, PNG, MP4, gzipped tarballs, and most PDFs do not shrink. Every one that reaches the compressor costs CPU and yields a ratio near 1.0. The fix is restricting compression to text content types with compression type.

Compressing tiny responses. Small JSON API responses compress poorly and sometimes grow. If comp_in / comp_rsp is small, set compression minsize-res <size> (available since HAProxy 3.2) so small payloads bypass the compressor and land in comp_byp where they belong. On versions without this directive, exclude small payloads by content type instead.

CPU-bound, TLS-terminating proxy. This is where compression is most dangerous. If the proxy terminates TLS, handshakes already dominate the CPU budget, and compression compounds that under traffic growth. On such a box, compression can be the difference between Idle_pct of 40 percent at peak and Idle_pct of 10 percent at peak, which is the difference between headroom and saturation. In CPU-constrained environments the right move is usually to disable compression.

Compression shed under load. HAProxy has self-protection knobs: maxcomprate caps the input rate to the compressor per process, and maxcompcpuusage (default 100, meaning no CPU limit) stops or reduces compression when CPU usage crosses a threshold. When these fire, the bypassed bytes land in comp_byp. This is good behavior: the proxy protects request processing by giving up bandwidth savings. But if your bandwidth plan assumes compression is active, a sustained comp_byp rise during peaks means your egress is silently growing.

Backend already compressing. If backend servers send responses with Content-Encoding already set, HAProxy does not re-compress them; those bytes show up in comp_byp and comp_in stays low relative to bout. Compression at the proxy is then redundant. Decide which layer owns compression and measure there.

Signals to watch in production

Signal	Why it matters	Warning sign
comp_out / comp_in	Compression effectiveness	Ratio near 1.0: paying CPU for no bandwidth gain
comp_byp trend vs comp_in	Whether compression is shed under load	comp_byp rising with flat or falling comp_in during peaks
comp_in / comp_rsp	Average payload size being compressed	Very small: tiny responses where compression is pure overhead
Idle_pct	Event-loop headroom that compression consumes	Falling toward 20 percent with compression enabled; below 5 percent is saturation
bout (bytes out)	Actual egress after compression	Egress climbing during peaks while the ratio stays good: bypass is active

The correlation that matters most is Idle_pct against the compression counters. If Idle_pct is falling and the compression ratio is poor, compression is a prime suspect and disabling it is a cheap, reversible experiment. If Idle_pct is falling and the ratio is good, compression is doing its job and the pressure is coming from elsewhere; the TLS handshake rate is the usual suspect, so check SslFrontendKeyRate from show info.

Two caveats. If busy-polling is enabled, Idle_pct sits near zero by design and tells you nothing; use run-queue depth from show activity or system CPU instead. And like all CSV counters, the compression counters reset on reload, so ratio math must handle resets.

How Netdata helps

Netdata collects the HAProxy stats CSV continuously, so comp_in, comp_out, comp_byp, and comp_rsp become time series rather than point-in-time snapshots. You see the bypass fraction climb during a peak instead of discovering it afterward.
Correlating the compression counters with Idle_pct on the same dashboard makes the CPU-versus-bandwidth trade concrete: you can see how much event-loop headroom compression costs at your real traffic levels.
Per-second collection catches short bursts where compression shedding is transient. A one-minute scrape interval can average away the exact peaks where comp_byp spikes.
Counter resets on reload are handled, so compression ratios survive config reloads without phantom drops.
Combining bout with comp_out shows real egress versus compressor output, which is how you verify whether the bandwidth savings you configured for are actually being delivered.

The Netdata solution

HAProxy load balancer monitoring with Netdata

Netdata monitors HAProxy with per-second frontend, backend, and queue metrics plus ML-powered anomaly detection. Correlate maxconn saturation, queue buildup, health-check cascades, 5xx attribution, and file-descriptor exhaustion against the backend and host signals behind them, so you catch the incidents in these runbooks before they page anyone.

See load balancer monitoring → Start monitoring free