vSphere monitoring checklist: the signals every host, VM, and vCenter needs

Send this to someone standing up vSphere monitoring for the first time, or rebuilding an alerting setup that pages too often and misses real incidents. It lists the signals worth collecting across the hypervisor plane (ESXi hosts and VMs) and the management plane (vCenter Server Appliance).

vSphere does not fail like a generic Linux box. CPU contention is invisible from inside the guest. Memory goes from fine to catastrophic in minutes once host swapping starts. A datastore at 99% full looks identical to one at 5% full from inside a VM, until every VM on it halts. And vCenter can degrade for weeks before anyone notices, because DRS, HA, and the API quietly keep working until they don’t. Generic CPU/disk/network dashboards miss most of this.

Use this checklist as a ladder. Survival is the floor: it tells you when something is dead. Operational catches the incidents that cause 3 a.m. pages. Mature and Expert add depth that experienced operators add after painful lessons. Skip levels at your own risk; a team that jumps straight to per-NUMA-node CPU tracking without monitoring host swap is going to have a bad time.

The two planes: hypervisor and management

A vSphere environment has two interdependent planes, and you must monitor them separately. Signals from one plane do not substitute for signals from the other.

Hypervisor plane (per ESXi host and per VM). This is where workloads run. The VMkernel scheduler arbitrates CPU, the four-tier memory reclamation cascade (TPS, balloon, compression, host swap) decides who gets punished when memory is tight, and the storage I/O path queues VM disk traffic. Most performance incidents live here.

Management plane (vCenter Server Appliance). This is the control plane. vpxd and an embedded PostgreSQL (vPostgres) hold inventory, tasks, events, alarms, and stats. When vCenter degrades, you lose DRS, HA monitoring, vMotion orchestration, provisioning, and visibility. Running VMs keep running, but you can no longer manage them.

flowchart TD
  subgraph Mgmt["Management plane: vCenter/VCSA"]
    M1["SDK/api reachable"]
    M2["vpxd + vPostgres up"]
    M3["/storage/* partitions"]
    M4["Cert expiry (STS + machine SSL)"]
    M5["DRS + HA cluster health"]
  end
  subgraph Hyp["Hypervisor plane: per host + per VM"]
    H1["Host connected + VM power state"]
    H2["CPU ready, co-stop, max-limited"]
    H3["Balloon, compression, swap"]
    H4["DAVG/KAVG/GAVG + queue depth"]
    H5["Snapshot age + datastore free"]
  end
  M2 -.depends on.-> M3
  M4 -.breaks.-> M1
  M2 -.manages.-> H1
  M5 -.reads.-> H2
  M5 -.reads.-> H3

Survival: is it up?

The absolute minimum. With only these, you know when something is dead but you are blind to degradation.

Ship only these six signals and you catch the most common outages: host down, datastore full, vCenter down, certificate expired. You miss everything that degrades slowly.

Operational: the signals that catch real incidents

This is the level where a competent team can actually diagnose a production incident. Grouped by domain.

Host and VM compute

Convert ready and co-stop summation counters to a percentage: value_ms / (interval_ms * vCPU_count) * 100. The raw counter is in milliseconds and rolls up, which hides spikes. A VM at 0% ready for 4.5 minutes and 50% for 30 seconds looks like 5% in a 5-minute rollup.

The vCPU oversizing trap is the most common mistake at this level. Adding vCPUs increases ready time because the scheduler must find more simultaneously-free pCPUs. A 16-vCPU VM on a host at 65% utilization can still have severe ready and co-stop. Co-stop is the smoking gun.

Memory

Memory is cliff-edge. A host at 85% consumed might be fine, or it might be in crisis. Percentage tells you nothing.

Watch for the cascade: balloon rises, then compression, then swap-out, then swap-in. Once swap-in is non-zero you are in an active emergency on every VM on the host. Swap I/O also competes with VM disk I/O on the datastore, which feeds back into storage latency.

Storage

The DAVG/KAVG split tells you where the problem lives. High DAVG with normal KAVG means the array or IP storage network is the bottleneck. High KAVG with normal DAVG means the VMkernel is queuing: SCSI reservation conflicts, VMFS metadata locking, or queue depth saturation. Treat NFS datastores as a special case, because DAVG includes network latency to the NFS server.

The snapshot time bomb is the single most common preventable incident. A snapshot left for weeks grows a delta VMDK that consumes the same datastore as the base disk, and consolidation of a large delta stuns the VM. VMware recommends no more than 2-3 snapshots per chain and no snapshot older than 72 hours.

Network

Availability and HA

HA uses both network and datastore heartbeats. If a host loses network heartbeats but datastore heartbeats continue, the host is “isolated” not “dead,” and the isolation response policy decides what happens next. Always know your isolation response. “Leave powered on” avoids split-brain but leaves VMs on the isolated host unmanageable.

vCenter itself

vCenter is infrastructure. Monitor it like infrastructure.

Critical detail on certificates: the STS signing certificate is separate from the machine SSL certificate. The STS cert is not visible in a browser, has its own lifecycle, and causes complete authentication failure when it expires. Check it with checksts.py (the STS certificate checker script published in VMware’s knowledge base), not with openssl s_client against port 443. Starting in vCenter 8.0 U3h, Machine SSL certificates issued by VMCA may auto-renew shortly before expiry when VMCA certificate mode is set, but this does not cover every certificate type. The STS cert must still be checked manually on current releases.

Critical detail on disk: the VCSA has roughly ten dedicated /storage/* mounts. /storage/log fills fastest when a service is error-looping, because errors generate more logs, which fill the partition, which causes more services to fail. /storage/db filling crashes vPostgres, which is immediately catastrophic.

Mature: signals for proactive teams

Add these once Operational is solid. They give 30 to 60 minutes of warning before most outages.

• Per-service CPU and memory on the VCSA (vpxd, vPostgres, STS, rhttpproxy), not just overall appliance utilization. Per-service data points to which subsystem is the bottleneck.
• vPostgres internals: connection count vs max_connections, dead tuple ratio on major tables, WAL directory size, longest running transaction. Dead tuples above 30% on a major table means autovacuum is losing.
• VCSA VM-level metrics from the hypervisor. A VCSA at 0% internal CPU with 50% CPU ready at the host is severely degraded. Memory balloon, swap, and datastore latency at the VM level are invisible from inside the guest.
• Task queue depth and average task duration. VM power-on taking 30 seconds instead of 10, or vMotion taking 30 minutes instead of 5, is the user-facing pain metric.
• Statistics rollup lag. If the 5-minute rollup takes more than 4 minutes, it is about to overlap the next interval. Historical charts go stale and the rollup itself creates I/O pressure.
• CPU co-stop and max-limited per VM. Add co-stop to every multi-vCPU VM.
• NUMA locality per VM for databases and in-memory caches.
• Memory compression rate per host as an early warning before swap.
• Network dropped packets and per-uplink utilization individually, not aggregated.
• VCHA replication state and lag if you run vCenter HA.
• vCLS health. If vCLS VMs are powered off or deleted, DRS stops working. Check under Cluster > Monitor > vSphere Cluster Services.
• Storage path health and APD/PDL events. A single remaining path to a production datastore is a TICKET; APD or PDL is a PAGE.

Expert: signals earned the hard way

Operators add these after a specific incident.

• Per-VM SCSI sense codes. Reading them resolves storage incidents hours faster.
• SCSI reservation conflicts per LUN for VMFS metadata lock contention.
• Per-VM CPU system vs user time. High system time often points to VMkernel overhead from NSX or storage drivers.
• vMotion network and vSAN network latency via vmkping.
• Guest OS time drift inside VMs after vMotion or snapshot revert.
• Per-NUMA-node CPU utilization to catch asymmetric load.
• VMkernel log error rate (/var/log/vmkernel.log) for early hardware and driver failure.
• STS Java heap utilization and GC frequency.
• vPostgres checkpoint frequency and autovacuum cadence per table.
• vmon restart counts per service to catch silent crash loops that external probes miss.
• SDK session count by client IP to find the misbehaving backup or monitoring integration hammering the API.
• Path state change rate to detect path thrashing before it becomes an outage.

What most teams get wrong

• Monitoring guest CPU instead of host CPU ready. A VM at 30% guest CPU with 15% ready is running at roughly 85% of requested speed. The guest has no idea.
• Treating memory as a percentage. A host at 85% consumed might be fine, or might be actively swapping. Monitor balloon, compression, and swap as independent reclamation indicators.
• No snapshot monitoring. Snapshots are the number one preventable cause of datastore-full incidents. Monitor count and age daily.
• Blaming the array when KAVG is the problem. DAVG and KAVG must be monitored separately.
• Cluster averages instead of per-host data. A cluster averaging 60% can have one host at 90% and another at 30%.
• Not monitoring vCenter itself. Thousands of VM dashboards, zero on the VCSA disk and database.
• Missing CPU limits (max-limited). A limit-constrained VM shows low ready and high max-limited. Many environments have forgotten limits silently throttling production.
• NIC teaming treated as bandwidth aggregation. A failed uplink means all traffic shifts to the remaining link. Teaming does not aggregate per flow unless you use LACP with an appropriate hashing policy, and even then a single flow uses one link.
• HA “protected” with admission control disabled. “Protected” is aspirational. With admission control off, restart capacity is not guaranteed.
• Ignoring certificate expiration. It hits teams every two years like clockwork, on the STS signing cert, not the browser cert.
• Latency without queue context. High latency with empty queues points to a different cause than high latency with full queues.
• vSAN capacity blindness. vSAN degrades significantly above 70% capacity, unlike traditional arrays.

How Netdata helps

Netdata’s value in a vSphere environment is correlation across the two planes at per-second resolution, which catches the spikes vCenter’s 5-minute rollups hide. The netdata/go.d vSphere collector pulls host and VM performance counters from the vCenter API; VCSA service and filesystem health is collected via the standard Linux collector set once SSH or agent-based collection is enabled on the appliance.

• Per-second CPU ready, co-stop, and max-limited per VM, alongside host CPU utilization, lets you distinguish genuine overcommitment (high ready plus high host CPU) from vCPU oversizing (high ready plus moderate host CPU) from forgotten limits (low ready plus high max-limited).
• Memory cascade correlation: balloon, compression, and host swap on the same chart makes the death spiral visible before swap-in starts.
• DAVG, KAVG, and GAVG alongside queue depth per datastore identifies whether latency originates at the array, the VMkernel, or both.
• Anomaly detection on vpxd error rate and SDK response time surfaces slow degradation that precedes vCenter outages, well before thresholds trip.
• Certificate expiry tracking across all certificate types (including STS, not just machine SSL) prevents the most common preventable vCenter outage.
• Composite pattern detection for the memory cascade, storage latency cliff, and vpxd overload cascade, so you see the pattern forming instead of reacting to a single threshold.

Related guides

• How vSphere actually works in production: a mental model for operators
• vSphere monitoring maturity model: from survival to expert
• vSphere CPU ready time high (%RDY): VMs starved while the guest looks idle
• vSphere CPU co-stop high (%CSTP): the SMP vCPU co-scheduling penalty
• vSphere CPU limit hit (%MLMTD): the forgotten MHz cap that silently throttles a VM
• vSphere vCPU oversizing: why adding vCPUs made the VM slower
• vSphere NUMA locality low: wide VMs paying the remote-memory tax
• vSphere memory ballooning (MCTLSZ): the host is reclaiming guest RAM
• vSphere host swapping (SWCUR/SWW/s): hypervisor swap and the memory death spiral
• vSphere memory compression: the reclamation tier between balloon and swap
• vSphere active vs consumed vs granted memory: why the percentage lies
• vSphere memory reclamation cascade: balloon to compress to swap in minutes