vSphere vMotion stun time: the switchover pause that drops connections

vMotion promises live migration with no disruption. For most VMs, the final switchover pause is short enough that in-guest applications and network clients never notice. But “live” is not “instantaneous.” During the final switchover, the VM is stunned: its vCPUs stop executing while the last set of dirty memory pages and device state transfers from the source host to the destination. When that pause stretches past a second, TCP stacks reset, databases miss heartbeats, and clustered applications fail over.

The stun is the mechanism that makes live migration possible. You cannot move a running VM’s memory without freezing it, copying what changed since the last pre-copy iteration, and resuming on the other side. Stun duration is driven by two variables: how fast the VM dirties memory and how much bandwidth the vMotion network provides between source and destination. Under 1 second is normal. Between 1 and 3 seconds is marginal. Above 3 seconds, connections drop and clusters fail over.

How it works

flowchart TD
    A[Migration starts] --> B[Pre-copy phase: copy memory pages]
    B --> C{Dirty rate converging?}
    C -->|No| D[SDPS: micro-stun vCPUs to slow writes]
    D --> B
    C -->|Yes| E[Switchover stun: freeze VM on source]
    E --> F[Transfer remaining dirty pages and device state]
    F --> G[Resume VM on destination host]
    G --> H{Stun duration}
    H -->|under 1s| I[Normal: applications unaffected]
    H -->|1 to 3s| J[Connections at risk]
    H -->|over 3s| K[Drops and cluster failovers]

vMotion proceeds in phases. The stun is the only moment during migration where the VM is genuinely down. Everything before it is pre-copy work that happens while the VM keeps running.

Pre-copy phase. The VMkernel copies the VM’s memory from source to destination while the VM continues running. As the VM writes to memory, previously copied pages become stale. The VMkernel traces dirty pages and iteratively recopies them. Each iteration should transfer fewer pages than the last if the dirty rate is low enough relative to transfer bandwidth.

Convergence and SDPS. If the VM writes memory faster than the vMotion network can transfer pages, pre-copy never converges: each iteration dirties as many pages as it copies. To force convergence, ESXi uses Stun During Page Send (SDPS), which briefly micro-stuns vCPUs in microsecond-scale bursts to slow the VM’s write rate below the transfer rate. SDPS is not the switchover stun. It is a pre-copy convergence mechanism that trades small scheduling pauses during migration for a shorter final stun.

Switchover stun. Once the remaining dirty-page set is small enough to transfer quickly, the VMkernel freezes the VM on the source host. The checkpoint phase transfers the remaining dirty pages plus device state: CPU register state, virtual device state, pending I/O. The checkpoint typically accounts for 100 to 200 milliseconds of the total stun. Total vmDowntime for standard VMs is usually under 500 milliseconds.

Resume on destination. The VM resumes execution on the destination host. The source copy is discarded and the migration is complete.

Stun duration is effectively the time to transfer the remaining dirty-page set during switchover plus the device-state checkpoint, divided by available vMotion bandwidth. Two factors dominate:

• Remaining dirty pages at switchover. If pre-copy converged well, this set is small. If the VM has a very high write rate or the pre-copy ran for fewer iterations (large VMs, impatient migration), more pages remain.
• vMotion network bandwidth and RTT. A dedicated 10 GbE or 25 GbE vMotion network with low round-trip time keeps the transfer fast. RTT above 10 milliseconds increases vmDowntime because each page transfer round trip costs more.

vGPU VMs are a special case. The GPU framebuffer cannot be pre-copied in older vSphere versions because the GPU hardware does not expose dirty-page tracking for its memory. The entire framebuffer must transfer during the stun. On vSphere 6.7 with a 16 GB framebuffer over 10 GbE, stun times can exceed 100 seconds and hit the default timeout.

Where it shows up in production

Stun time is invisible until it crosses application tolerance thresholds. The symptoms appear at the application layer, not the hypervisor layer:

• TCP connection drops. Aggressive TCP keepalive or retransmission timeout settings may interpret a 2-to-3-second pause as a dead connection and reset. Load balancers with short health-check intervals may mark the backend as down.
• Database heartbeat misses. Clustered databases (Oracle RAC, SQL Server AlwaysOn, Galera) and distributed consensus systems (etcd, ZooKeeper) rely on heartbeat intervals. A stun longer than the heartbeat timeout can trigger failover, member removal, or split-brain.
• Application-level timeouts. Request-response protocols with tight timeouts (financial trading systems, some RPC frameworks) fail outright when the VM pauses for more than a second.
• Clock drift. The guest OS clock jumps forward by the stun duration. For short stuns this is corrected by NTP or VMware Tools time sync. Tens to hundreds of milliseconds of drift is normal after vMotion.
• Transient CPU ready spike. The migrating VM shows a brief spike in CPU ready or wait time during and immediately around the stun. Brief 5-to-10-percent spikes lasting under 30 seconds during DRS migrations are normal and expected.
• Snapshot operations compound stuns. Snapshot deletion and consolidation require their own stuns. A VM that is migrated and then has a snapshot consolidated in the same window experiences multiple stuns in sequence.

DRS thrashing amplifies all of these. If DRS repeatedly migrates a VM between hosts (aggressive settings, constraint conflicts, oscillating load), the VM accumulates cumulative stun time. Large VMs take longer to vMotion and stun longer at switchover because pre-copy must transfer more memory. A VM stunned 3 seconds per migration and migrated 5 times in an hour has lost 15 seconds of execution time, potentially spread across multiple application-sensitive windows.

When stun time becomes a problem

What pushes stun time higher:

• High memory dirty rate. Write-heavy workloads (in-memory databases, large analytics, active log buffers) dirty pages faster than pre-copy can track. The remaining dirty-page set at switchover is large. SDPS helps converge but cannot fully eliminate the residual set for extreme write rates.
• Insufficient vMotion bandwidth. A shared or undersized vMotion network (1 GbE, or 10 GbE shared with VM traffic and storage) cannot transfer the remaining page set quickly during switchover. Network I/O Control and traffic shaping can further throttle vMotion if misconfigured.
• High network RTT. The vmotionRTT field records round-trip time between source and destination. RTT above 10 milliseconds significantly increases vmDowntime because each page transfer costs a round trip. Cross-site vMotion over WAN or stretched-L2 links is the most common cause.
• Large VMs. More memory means more pages to pre-copy and a larger residual set at switchover. Large VMs also take longer to migrate, increasing the window during which DRS may initiate additional migrations.
• vGPU VMs. GPU framebuffers transfer during the stun on older vSphere versions. Even on VCF 9.0, large multi-GPU VMs (8x H100, 640 GB framebuffer) stun for tens of seconds without the IOMMU D-bit pre-copy path.
• DRS aggressiveness. DRS migration threshold levels 1 through 5 control how aggressively DRS migrates VMs. Level 3 is default. Higher levels trigger more migrations, increasing the probability that a sensitive VM is migrated during a load-sensitive window. DRS runs every 5 minutes by default, so migration bursts can cluster.

Signals to watch in production

To extract vmDowntime from logs, grep the destination host’s hostd.log for the migration result.

# On the destination ESXi host (SSH or DCUI shell):
grep -i "stun\|Migration.*complete\|vmDowntime" /var/run/log/hostd.log

The vmDowntime field in the VmotionResult struct gives total downtime in microseconds. A related field, vmStunTime, often reports 0 and is not reliable; use vmDowntime.

For vGPU VMs, setting the vMotion Stun Time Limit lower than the estimated maximum stun time may prevent the VM from powering on. Estimate stun time before adjusting the limit.

How Netdata helps

• Correlate CPU ready spikes with vMotion events. Netdata’s per-second CPU metrics for vSphere VMs let you see the brief ready-time spike that accompanies a stun and correlate it with the migration timestamp, confirming the spike is transient vMotion overhead rather than sustained host contention.
• Surface DRS migration rate over time. Tracking migration frequency against host CPU and memory utilization disparity shows whether DRS is balancing effectively or thrashing, which compounds stun exposure.
• Watch host-level memory pressure during migration. A brief balloon on the target host as it makes room for the incoming VM is normal. Sustained ballooning or swap after migration indicates the VM landed on an overcommitted host.
• Detect application-layer impact. If Netdata monitors the guest OS or application (database, web server, message queue), correlate connection drops, heartbeat misses, or latency spikes with vMotion events to confirm stun time as the cause.
• Track vMotion network utilization. vMotion consuming an entire uplink during migration is expected. Sustained saturation across the vMotion NIC team indicates bandwidth constraints that will lengthen stun times.

Related guides

• vSphere CPU ready time high (%RDY): VMs starved while the guest looks idle
• vSphere CPU co-stop high (%CSTP): the SMP vCPU co-scheduling penalty
• vSphere memory ballooning (MCTLSZ): the host is reclaiming guest RAM
• vSphere host swapping (SWCUR/SWW/s): hypervisor swap and the memory death spiral
• How vSphere actually works in production: a mental model for operators