vSphere DRS thrashing: vMotion churn with no stable placement

DRS thrashing is when the Distributed Resource Scheduler cannot find stable placement and keeps migrating the same VMs between hosts. Each migration costs vMotion bandwidth and inflicts a brief stun on the VM. The cluster looks balanced on paper, but the migrations never stop.

This is almost always the DRS cost-benefit model oscillating because something underneath is unstable: bursty CPU or memory pressure, an aggressive migration threshold, conflicting affinity rules, or capacity too tight to ever be balanced.

DRS evaluates placement periodically, runs a cost-benefit analysis, and recommends vMotion when the projected benefit exceeds the projected cost. When conditions flip every cycle, DRS moves the VM back. Migration rate climbs, the vMotion network saturates, and large or memory-dirty VMs accumulate stun time. The symptom is “DRS keeps moving the same VMs,” and the fix is rarely to make DRS more aggressive.

What this means

DRS is not real-time. By default it evaluates placement every five minutes on vSphere 6.x. vSphere 7.0 rewrote the algorithm and runs load balancing every one minute.

Each cycle, DRS asks: would moving VM X from host A to host B reduce cluster imbalance enough to justify the vMotion cost? When the answer oscillates, you get thrashing.

The memory side is also version-dependent. vSphere 7.0+ uses Granted Memory (host physical memory mapped to the VM) to compute host memory utilization, replacing Active Memory. The explicit goal was to reduce vMotions triggered by Active Memory fluctuations. There is no option to revert. If you are still on 6.x, Active Memory fluctuations are a more likely thrashing trigger.

The migration threshold slider has five positions. Level 3 is the default. Level 5 (Aggressive) tells DRS to recommend vMotions for even slight imbalance; Level 1 (Conservative) only recommends mandatory priority-1 moves. A cluster at Level 5 with bursty workloads is the textbook recipe for churn.

flowchart TD
  A[Workload oscillates per cycle] --> B[DRS cost-benefit flips]
  B --> C[Same VM migrated back and forth]
  C --> D[vMotion bandwidth consumed]
  C --> E[VM stun per migration]
  D --> F[Other vMotion-dependent ops slow]
  E --> G[Per-VM stun accumulates]
  F --> H[Cluster appears unbalanced despite moves]
  G --> H
  H --> A

The loop feeds itself. The cluster never converges because the signal DRS reacts to keeps changing every evaluation cycle.

Common causes

Quick checks

These are safe read-only checks.

# PowerCLI: count DRS-initiated migrations in the last hour
Get-VIEvent -Entity (Get-Cluster <cluster>) -Types "DrsVmMigratedEvent" -Start (Get-Date).AddHours(-1) | Measure-Object

# PowerCLI: DRS configuration
Get-Cluster <cluster> | Select Name, DrsEnabled, DrsAutomationLevel

# PowerCLI: failed migrations (infrastructure issues masquerading as thrashing)
Get-VIEvent -Types "VmFailedMigrateEvent" -Start (Get-Date).AddHours(-24)

# PowerCLI: pending DRS recommendations
Get-DrsRecommendation -Cluster <cluster>

# PowerCLI: rule violations and DRS faults
Get-VIEvent -Entity (Get-Cluster <cluster>) -Start (Get-Date).AddHours(-1) | Where-Object {$_.EventTypeId -match "Drs|compliance|rule"}

# esxtop: vMotion vmknic bandwidth utilization (press 'n' for network view)
esxtop

How to diagnose it

1. Quantify the churn. Pull DrsVmMigratedEvent count per hour for the last 24 to 48 hours. Thresholds worth investigating: more than about 10 migrations per hour in a stable cluster, or migration rate greater than 2x the 7-day average. Cyclical patterns where the same VMs migrate back and forth across cycles are the clearest thrashing signal.

2. Identify the recurring VMs. Group migration events by VM. A handful of VMs accounting for most migrations is the strongest thrashing signature. Record their vCPU count, configured memory, and workload type (database, batch, VDI, build agent).

3. Compare host utilization across the cluster. If every host sits above 85% CPU or consumed memory, there is no balanced placement possible, and DRS will keep trying. If hosts oscillate between, say, 60% and 90% within a single DRS evaluation window, the workload is too bursty for the current threshold.

4. Check the DRS migration threshold. Anything above Level 3 in a volatile environment is suspect. Note that PowerCLI displays the threshold value inverted relative to the vSphere UI slider: in the API a lower number means more aggressive. Do not change it yet.

5. Check affinity and anti-affinity rules. DRS rules constrain placement. Conflicting or overly strict rules can prevent DRS from ever reaching a low imbalance score. This looks identical to thrashing from the migration rate alone.

6. Check vMotion network bandwidth. Use esxtop network view on the vmknic tagged for vMotion. Sustained near link capacity during steady-state operation (not during maintenance mode) means thrashing is starving legitimate migrations.

7. Check vCLS VM health on vSphere 7+. DRS depends on vSphere Cluster Services. Unhealthy vCLS VMs raise the “vSphere DRS functionality was impacted due to unhealthy state vSphere Cluster Services” alarm and degrade or block migrations.

8. Check per-migration stun time. Large or memory-dirty VMs stun longer per move. Per-VM stun accumulation across a day is usually the real user-visible cost, even when the raw migration rate looks only mildly high.

Metrics and signals to monitor

Fixes

Group fixes by the underlying cause. Apply the cheapest, safest one first.

Lower the migration threshold

For most bursty-workload thrashing, dropping the DRS migration threshold from Level 5 to Level 3, or from Level 3 to Level 2, is the single most effective change. DRS will recommend fewer migrations because the cost-benefit bar is higher.

This takes effect on the next DRS evaluation cycle and is non-disruptive. Watch the migration rate for the next hour.

Address capacity headroom

If hosts are uniformly above 85% CPU or memory, DRS has no good answer. Options:

• Move some VMs to a different cluster.
• Add a host, restoring N+1 headroom.
• Right-size oversized VMs to free real capacity.
• Add memory reservations on critical VMs to prevent memory-driven moves.

Right-sizing is the highest-leverage option but requires a VM power cycle.

Soften affinity rules

Review affinity and anti-affinity rules. “Must” rules are hard constraints. “Should” rules let DRS violate them when balance is poor. Converting “must” to “should” for non-critical rules gives DRS room to find a stable placement.

Pin specific thrashing VMs

If one or two VMs repeatedly migrate, consider VM-Host affinity to a specific host, or partial CPU and memory reservation, to take them out of the placement pool. This is a targeted fix, not a global one.

vGPU-aware DRS (vSphere 8.0 U2+)

If the thrashing involves vGPU-enabled VMs, vSphere 8.0 Update 2 added vGPU-aware DRS tuning that estimates vMotion stun time based on framebuffer size and network bandwidth. List the vGPU-aware DRS advanced options on your cluster (for example with PowerCLI Get-AdvancedOption) and confirm they are configured before assuming vGPU VMs are eligible for ordinary DRS moves.

Do not restart vCenter as a fix

Restarting vpxd drops all in-flight tasks, takes 5 to 15 minutes to rebuild the inventory cache, and after restart DRS triggers a burst of catch-up vMotions that makes thrashing worse for 1 to 2 hours.

Prevention

• Right-size VMs before adding capacity. Oversized VMs are harder for DRS to schedule and more disruptive to migrate.
• Track the 7-day migration baseline. Without a baseline you cannot tell normal cluster noise from thrashing.
• Keep DRS at Level 3 unless you have a specific reason. Level 5 in a bursty environment is the most common thrashing pattern.
• Reserve the vMotion network for vMotion. Sharing the migration network with VM or storage traffic produces both slow migrations and slow storage.
• Treat vCLS VMs as infrastructure on vSphere 7+. DRS depends on them and so do you.
• Watch per-VM stun accumulation, not just migration count. A few large-VM migrations can cost more than many small-VM migrations.
• Check DRS rules during every major change. New affinity rules added for one workload can silently destabilize placement for unrelated workloads.

How Netdata helps

• Per-second DRS migration rate and per-VM migration count, so you can see the oscillation pattern that 5-minute rollups hide.
• vMotion stun time correlated with per-VM vCPU and memory size, which tells you whether churn is concentrated on a small number of large VMs.
• Host CPU and consumed versus active memory per host side by side, so you can tell whether DRS even has a balanced placement to reach.
• vMotion vmknic bandwidth utilization, so you can detect when thrashing is saturating the migration network and starving legitimate maintenance vMotions.
• ML-based anomaly detection on the migration rate that flags the cost-benefit oscillation pattern even when no single metric crosses a hard threshold.
• Correlation between DRS migration spikes and CPU ready or co-stop spikes on the migrating VMs, which distinguishes thrashing from healthy balancing.

Related guides

• vSphere active vs consumed vs granted memory: why the percentage lies
• vSphere CPU co-stop high (%CSTP): the SMP vCPU co-scheduling penalty
• vSphere CPU limit hit (%MLMTD): the forgotten MHz cap that silently throttles a VM
• vSphere CPU ready time high (%RDY): VMs starved while the guest looks idle
• vSphere datastore full: ‘No space left on device’, paused VMs, and power-on failures
• vSphere datastore IOPS and throughput: spotting storage saturation before latency bites
• vSphere datastore latency high: reading GAVG, DAVG, and KAVG
• vSphere dropped packets (%DRPRX/%DRPTX): ring buffers, CPU, and uplink backpressure
• vSphere storage latency cliff: the ’everything is slow’ incident that hits every VM at once
• vSphere ESXi hardware health: ECC errors, fan failure, and thermal throttling
• vSphere host ‘Not Responding’: dead, isolated, or is hostd hung?
• vSphere PSOD (purple screen of death): diagnosing an ESXi host crash