vCenter SSO intermittently failing: STS heap, flaky AD, and near-expiry certs

Users report “vCenter logged me out randomly.” PowerCLI jobs fail with a token validation error, then succeed on the second attempt. One host shows “Not Responding” for 90 seconds, then comes back. Ten minutes later a different host does the same. The web client sometimes loads, sometimes returns a 503. STS itself sits at “Started/Green” in vmon the whole time.

This is the SSO/STS silent degradation pattern. STS is not down. It is validating some SAML token requests and failing others. Every individual failure looks transient, every retry works, and by the time someone escalates the symptom has moved to a different user or host.

Three root causes account for almost all of these incidents. STS Java heap running near max, with full GC pauses timing out token validation requests. A flaky Active Directory identity source where some LDAP lookups succeed and others time out. And an STS signing certificate close enough to expiry that clients with even a small amount of clock skew start rejecting tokens. They also combine: heap pressure under load plus AD slowness produces failures that do not map cleanly to any single cause.

What this means

STS (vmware-stsd) is a Java service that issues and validates SAML tokens for every authenticated operation in vCenter. Anything that makes individual token requests slow, inconsistent, or untrustworthy produces the same user-visible symptom: some logins and API calls work, others do not.

The defining characteristic is non-deterministic failure. A consistent 100% failure rate means STS is broken, and that is usually obvious. An intermittent failure rate, where the same user authenticates twice and only the second attempt works, means STS is degraded in a way the average health check will not surface. The vmon “GREEN” status only proves the service process is alive and answering its liveness probe, not that it is validating tokens within latency budget.

flowchart TD
  A[Intermittent SSO failure] --> B[STS heap near max]
  A --> C[Flaky AD identity source]
  A --> D[Near-expiry STS cert]
  B --> B1[Full GC pauses time out token requests]
  B --> B2[Heap RSS near Xmx, swap non-zero]
  C --> C1[Some LDAP lookups succeed, others time out]
  C --> C2[AD-integrated logins fail, vsphere.local works]
  D --> D1[Clock skew pushes clients past token validity edge]
  D --> D2[Hosts and integrations with stale time reject tokens]

The three branches share a common surface (intermittent auth failures) but diverge in the logs and checks that expose them, so diagnosis requires looking at more than one signal at once.

Common causes

Quick checks

Run these read-only checks first. All are safe on a production vCSA.

# Check STS service state and health
/usr/lib/vmware-vmon/vmon-cli --status vmware-stsd

# STS heap and GC frequency from the runtime log
grep -i "GC\|garbage\|heap" /var/log/vmware/sso/sts-runtime.log | tail -50

# Per-Java-process RSS and swap (vCSA runs multiple JVMs; identify STS by cmdline)
for pid in $(pidof java); do
  echo "--- PID $pid ---"
  tr '\0' ' ' < /proc/$pid/cmdline | head -c 160; echo
  grep -E "VmRSS|VmSwap" /proc/$pid/status
done

# NTP offset on the vCSA itself
chronyc tracking

# Enumerate domain controllers vCenter is aware of
/opt/likewise/bin/lw-get-dc-list <domain-name>

# STS signing certificate expiry
python /usr/lib/vmware-vmca/bin/checksts.py 2>/dev/null || echo "checksts.py not present; use vCert"

# Token validation error rate in the last hour
grep -c "token.*validat\|InvalidTimeRange\|not authenticated" /var/log/vmware/sso/sts-runtime.log

# SSO authentication failures with context
grep -E "LOGIN_FAILED|Authentication.*failed" /var/log/vmware/sso/vmware-sts-idmd.log | tail -20

# vpxd-side TLS errors that suggest cert rejection by hosts
grep -iE "ssl|certificate|tls" /var/log/vmware/vpxd/vpxd.log | tail -30

checksts.py is the historical tool referenced throughout the STS playbook and works for a read-only expiry check. Broadcom has deprecated it on current builds in favour of the vCert tool (KB 385107), which is the supported replacement. Either works for inspection; use vCert for any actual replacement workflow.

How to diagnose it

Work through these in order. The order matters because heap and AD issues are reversible under load, while an expired or nearly-expired STS cert is a fixed-date problem that will only get worse.

1. Confirm the pattern is intermittent, not total. If administrator@vsphere.local can log in but AD users cannot, the problem is the AD identity source, not STS itself. If everyone fails 100% of the time, STS is broken, not degraded. Look at vmware-sts-idmd.log for LOGIN_FAILED and LDAP errors to separate the two.

2. Pull STS GC behaviour from sts-runtime.log. Frequent Full GC entries, especially with long pause times immediately before token validation timeouts, indicate heap pressure. Cross-check with the per-process RSS from the quick checks. A Java service whose RSS sits at or near its configured -Xmx and whose VmSwap is non-zero is under heap pressure. On vCSA, any swap usage on a Java service is abnormal.

3. Check the AD identity source independently of STS. Use /opt/likewise/bin/lw-get-dc-list <domain-name> to enumerate the domain controllers vCenter is aware of. Then run an explicit LDAP bind and search against each one. If one DC times out and another responds quickly, STS will intermittently fail AD lookups depending on which DC it hit. The fix path is to reconfigure the identity source to point at a known-good DC, or to fix the slow DC.

4. Check the STS signing certificate. STS uses an internal signing cert that is separate from the machine SSL cert visible in the browser. The most common operator mistake is checking only the machine cert. Use checksts.py or vCert’s certificate-info option to inspect the STS signing cert. Anything inside 30 days of expiry is the warning zone; anything inside 7 days is urgent.

5. Check NTP on the vCSA and on the failing clients. SAML tokens have a validity window defined by clock time. If the vCSA and a client disagree by even a few minutes, that client may reject tokens that other clients accept. chronyc tracking on the vCSA should show a low offset. Clients with their own clock skew produce intermittent failures that look exactly like a cert problem.

6. Look for combinations. A near-expiry STS cert plus clock skew on a subset of hosts explains host flapping. Heap pressure plus AD slowness explains why failures cluster during peak login hours. In practice two causes overlap more often than not.

Metrics and signals to monitor

Fixes

STS heap pressure

First, confirm there is no underlying cause inflating heap usage. A known trigger is a very large CRL being parsed by the IdmCrlCachePeriodicChecker thread when smart card authentication is configured. In that case the heap bump is a workaround, not a fix. The underlying fix is to move revocation checking from CRL to OCSP, or to obtain a smaller CRL from the CA.

To increase the STS heap to 1024 MB:

# Configuration change: increases STS heap reservation. Takes effect on service restart.
cloudvm-ram-size -C 1024 vmware-stsd

The VCSA VM must have enough total RAM that other services are not starved. If the VCSA is sized at or near its memory limit, increase the VM’s memory first, then bump the STS heap. Do not restart STS as the first action: restarting drops in-flight token requests and will worsen the apparent outage for the window it takes the service to come back and rebuild state.

If the heap pressure is from CRL parsing and OCSP is not an option, the heap increase is the supported workaround. If it is from a general leak in a specific build, patch to a fixed version rather than continuously bumping heap. KB 415856 documents a known vc-ws1a-broker memory leak on vCenter 8.0 U2a and later that exhausts swap and pushes STS and other Java services into intermittent red/green flapping. Restarting vc-ws1a-broker is a temporary fix; patching is the durable one.

Flaky AD identity source

Identify which DC is slow. The first response that works is not proof the identity source is healthy, because STS rotates through DCs.

/opt/likewise/bin/lw-get-dc-list <domain-name>

For each DC returned, run an explicit LDAP bind and a small read-only search. This authenticates to the DC with a real bind user, so use a low-privilege read-only service account:

ldapsearch -H ldaps://<dc-fqdn>:636 -D '<bind-user>' -W -b '<base-dn>' '(sAMAccountName=<test-user>)'

A DC that takes seconds to respond, or that times out, is the source of the intermittent failures. Reconfigure the vCenter identity source to point at a specific known-good DC rather than resolving to the domain. The domain-level lookup is what allows STS to hit a slow DC.

Two related AD failure modes worth ruling out when AD changes coincide with the start of the incident. If the LDAPS certificate on a DC was recently renewed, vCenter’s identity source may still hold the old certificate and every AD login will fail until the identity source is updated. And if the native platform error code in vmware-sts-idmd.log is 851968 alongside USER_NAME_PWD_AUTH_FAILED, that is the signature of STS hitting a slow or unresponsive DC rather than a genuine wrong-password.

Near-expiry STS cert

Use the vCert tool (KB 385107) to inspect and replace the STS signing certificate. The historical fixsts.sh and checksts.py replacement workflows are superseded by vCert on current builds. High-level steps:

1. Download vCert from KB 385107, unzip on the vCSA.
2. Run ./vCert.py and use the certificate-info option to confirm the STS signing cert expiry.
3. Use the STS replacement option to regenerate the signing cert. Read the KB procedure end to end before running it. Certificate replacement in Enhanced Linked Mode topologies affects all vCenter instances in the topology, not just the one you are working on.
4. Restart the services the KB specifies. Some services cache the old STS cert and will keep failing until restarted.

Before replacing, fix any clock skew on the vCSA and on hosts that were flapping. A renewed cert does not help if half the clients still reject tokens because their clocks are minutes off. Also check the STS_INTERNAL_SSL_CERT store in VECS for corruption. KB 422821 documents a case where a missing __MACHINE_CERT key in that store produces “No healthy upstream” symptoms that look like an STS cert problem but are actually a store-level issue.

Prevention

• Track STS signing certificate expiry separately from machine SSL certificate expiry. The STS cert has its own lifecycle and is not visible in the browser. Treat anything inside 60 days as in-progress work, not a future task.
• Treat STS heap usage as a leading indicator. A STS service consistently above 80% of Xmx will tip into Full GC pauses under load. Trend it.
• Monitor NTP offset continuously on the vCSA and on ESXi hosts. Time skew above a few seconds is a prerequisite for several authentication failure modes, not just this one.
• For AD-integrated deployments, monitor LDAP latency and timeout count per DC. The identity source is only as reliable as the slowest DC it can resolve to. Domain-level resolution hides per-DC variance.
• Patch the VCSA on a regular cadence. Several of the heap and swap issues that produce this pattern are fixed in specific builds, including the vc-ws1a-broker leak noted above.
• Alert on host flapping as a first-class signal. A host cycling connected/not responding/connected is rarely a host problem when multiple hosts do it within the same window. It is almost always vCenter-side, and often cert or time related.

How Netdata helps

• Per-second JVM and process metrics on the vCSA surface STS heap and GC pause behaviour at the resolution the intermittent failures actually happen at. Five-minute averages hide the GC spikes that time out token requests.
• Correlate STS heap pressure, SSO authentication failure rate, and NTP offset in one view. The silent-degradation pattern only makes sense when you can see all three signals over the same window.
• Track certificate expiry as a first-class metric with configurable lead-time alerts, so the STS signing cert is not discovered only when clients start rejecting tokens.
• ESXi host connection state changes and vCSA service health feed the same timeline, which makes host-flapping during a suspected cert or time-skew incident easy to separate from a genuine host-side problem.

Related guides

• vSphere active vs consumed vs granted memory: why the percentage lies
• vSphere CPU co-stop high (%CSTP): the SMP vCPU co-scheduling penalty
• vSphere CPU limit hit (%MLMTD): the forgotten MHz cap that silently throttles a VM
• vSphere CPU ready time high (%RDY): VMs starved while the guest looks idle
• vSphere datastore full: ‘No space left on device’, paused VMs, and power-on failures
• vSphere datastore IOPS and throughput: spotting storage saturation before latency bites
• vSphere datastore latency high: reading GAVG, DAVG, and KAVG
• vSphere dropped packets (%DRPRX/%DRPTX): ring buffers, CPU, and uplink backpressure
• vSphere DRS not balancing: affinity rules and reservations blocking placement
• vSphere DRS thrashing: vMotion churn with no stable placement
• vSphere storage latency cliff: the ’everything is slow’ incident that hits every VM at once
• vSphere HA host isolation and split-brain: when isolation response goes wrong