vSphere PSOD (purple screen of death): diagnosing an ESXi host crash

A purple screen of death (PSOD) is the ESXi VMkernel’s deliberate halt. When the kernel detects an unrecoverable condition, an uncorrectable machine check, a driver panic, or a corrupted data structure, it stops the host on purpose, paints the purple diagnostic screen, and writes a core dump if a target is configured. Every VM on that host dies instantly. There is no graceful shutdown and no live migration off the host.

The operational impact is binary and severe. HA detects the lost host within seconds and begins restarting protected VMs on surviving cluster members. Typical restart windows are 30 to 120 seconds, gated by admission control capacity, datastore accessibility, and VM restart priority. Anything not protected by HA, anything pinned to the host by affinity rules, and anything whose restart ordering is wrong stays down until an operator intervenes.

Your forensic evidence is short-lived. The purple screen itself disappears on reboot. The persistent evidence is whatever the host wrote to its core dump target before halting, plus a few log lines captured just before the panic. If no dump target was configured, you lose the most diagnostic data you will ever have for this incident. The single most common post-mortem mistake is resetting a PSODed host while the dump is still being written.

What this means

PSOD is not a guest crash, not a hostd hang, and not a vCenter communication failure. The VMkernel itself chose to stop because continuing would risk corruption. The host is gone until it reboots. VMs are gone until HA restarts them or you power them on manually.

A PSOD screen carries a fixed set of fields:

The backtrace is usually the most diagnostic field. A fault inside a named driver module, for example an FCoE driver or an iLO management driver, points you at that component. A fault inside generic kernel code with no driver name is more often hardware, especially when the exception type is a Machine Check Exception (MCE).

flowchart TD
    A[PSOD observed] --> B{Core dump captured?}
    B -- Yes --> C[Copy zdump, extract with vmkdump_extract]
    B -- No --> D[Configure dump target before next crash]
    C --> E[Search vmkernel-log for Bluescreen and MCE]
    E --> F{Hardware or driver signal?}
    F -- MCE or ECC errors --> G[Memory DIMM or CPU fault]
    F -- Named driver backtrace --> H[Driver or firmware mismatch]
    F -- Unclear --> I[Escalate to VMware support with dump]

Since vSphere 7.0, the default core dump target is a file on the VMFS-L ESX-OSData system volume rather than a dedicated diagnostic partition. Partition-based dumps are still supported but are no longer the default. Either target is acceptable; the failure mode is having no target at all. ESXi warns “No coredump target has been configured. Host core dumps cannot be saved” when this is the case. Do not suppress or ignore that warning.

Common causes

Quick checks

Run these read-only commands on the affected host after it reboots, or on a healthy host to verify dump target configuration before the next crash.

# Confirm configured dump target
esxcli system coredump file list
esxcli system coredump partition list
esxcli system coredump network get

# List any core dumps still on the host
ls -la /var/core

# Search vmkernel log for the panic and any hardware exceptions
grep -i "panic\|@Bluescreen\|MCE\|unexpected\|Machine Check" /var/log/vmkernel.log

# Search vobd (VMkernel Observation Daemon) for reboot and panic markers
grep -i "panic\|unexpected\|reboot" /var/log/vobd.log

# Hardware identity and memory state
esxcli hardware platform get
esxcli hardware memory get

# SMART data for any disk suspected in a storage driver panic
esxcli storage core device smart get -d <naa-device-id>

# ESXi build and patch level
vmware -vl

The @Bluescreen keyword in the extracted vmkernel log is the canonical marker that this boot followed a PSOD. vobd.log records host lifecycle events including the unexpected-reboot marker that confirms a crash happened even if nobody saw the screen.

How to diagnose it

1. Do not reset the host while the purple screen is displayed. The dump write can take minutes, and on large memory hosts it can take much longer. Resetting early destroys the only copy of the faulting state. Wait until the screen reports the dump completed or the host reboots itself.

2. Confirm the dump target was configured before the crash. If esxcli system coredump file list, partition list, and network get all return empty, you have no persistent evidence. Configure a target before the next crash. See Fixes below.

3. Copy the dump off the host. For a partition-based dump, copy with esxcfg-dumppart:

   # Copy the partition dump to a VMFS volume for analysis
   esxcfg-dumppart --copy --devname "/vmfs/devices/disks/<identifier>" \
     --zdumpname /vmfs/volumes/<DatastoreName>/<hostname>-<date>-zdump
   

   For a file-based dump on ESX-OSData, the dump file is already on the host’s filesystem. Either way, move the dump to storage that survives a host rebuild before you patch or reinstall.

4. Extract and search the dump text. Use vmkdump_extract to pull the embedded vmkernel log out of the zdump, then search for the panic marker:

   vmkdump_extract vmkernel-zdump.1
   grep -i "@Bluescreen\|panic\|MCE" vmkernel-log.1
   

5. Read the backtrace. Identify whether the fault is inside a named driver module or in generic kernel code. A named driver narrows the search dramatically.

6. Correlate with hardware sensors. An MCE with no driver in the backtrace, combined with rising correctable ECC counts in the hours before the crash, points to memory or CPU. Check esxcli hardware memory get and the vendor BMC (iLO, iDRAC, XCC).

7. Check the driver and firmware baseline. Cross-reference every installed driver VIB and firmware version against the VMware Compatibility Guide. Mismatched versions, especially between companion drivers such as nmlx5_core and nmlx5_rdma, are a known PSOD source.

8. Check patch level against current CVEs. Three ESXi zero-days (CVE-2025-22224, CVE-2025-22225, CVE-2025-22226) were patched in early 2025 and were exploited in the wild. Compare the host’s build against the vendor’s current security bulletin before assuming the cause is hardware.

9. Escalate to VMware support with the dump if the cause is unclear. The full dump, plus vm-support output from the host after reboot, is what support needs. Opening a support request without the dump usually means starting over after the next crash.

Metrics and signals to monitor

Fixes

No dump target configured

Configure one before the next PSOD. The simplest option for a cluster is a network dump collector:

# Configure network dump collector
esxcli system coredump network set --interface-name vmk0 \
  --server-ip <collector-ip> --server-port 6500
esxcli system coredump network set --enable true

The vCenter VAMI ships an ESXi Dump Collector service that listens on port 6500 by default. It is disabled out of the box; set it to Automatic and start it. Alternatively, configure a file-based or partition-based local target via esxcli system coredump file set or partition set.

The --server-ipv4 flag is deprecated on current ESXi builds in favor of --server-ip.

Hardware fault (memory or CPU)

Run the vendor’s offline hardware diagnostics. If a specific DIMM is flagged, replace it. If the pattern is varying backtraces with no consistent driver, suspect the CPU. Hardware replacement is the only real fix. Driver and firmware updates will not mask a failing DIMM or socket, and chasing a software cause on failing hardware burns days.

Driver or firmware mismatch

Update the offending driver VIB to the version listed in the VCG for your hardware and ESXi build. Apply the matching firmware at the same time. A driver update without the firmware update, or vice versa, often leaves you on an untested combination. Known patterns include HPE iLO Native Driver leaks (fixed in iLO Native Driver 10.8.2 and later, plus ESXi 8.0 Update 2b), Marvell or QLogic FCoE driver faults, and Mellanox nmlx5 driver version mismatches after upgrade where the nmlx5-rdma VIB is missing.

Outdated BIOS or CPU microcode

Update the server BIOS to the latest vendor version. ESXi 8.x in particular expects a microcode level the older BIOS images do not deliver. Booting 8.x on a stale BIOS can produce repeatable #PF Exception 14 PSODs with a MicrocodeUpdate warning. If the PSOD persists after a BIOS update, test RAM for physical defects.

vMotion race condition

If the crash consistently follows a vMotion on ESXi 7.0 U2/U3 or 8.0 GA, set sched.mem.migPreval.enable = "FALSE" on affected VMs as a workaround, and plan an upgrade. This race is reportedly fixed in ESXi 7.0 U3l and 8.0b. Until you are on a fixed build, consider adding the setting to VM template defaults for memory-heavy workloads.

Unpatched ESXi CVE

Apply the current patch baseline. Treat CVE-driven PSODs as a security incident, not just a stability issue. The same exploitation path that crashes one host may already be in use against the wider estate.

Prevention

• Configure a network dump collector on every host, not just the ones that have already crashed.
• Run a daily check that every host has a reachable, configured dump target. An empty dump configuration is a silent failure that only surfaces at the worst possible moment.
• Track host uptime and alert on unexpected reboots that were not part of a planned change.
• Monitor CIM hardware health continuously, with special attention to correctable ECC rates on memory.
• Maintain a VCG-baseline driver and firmware report. Drift from the tested combination is the most common preventable PSOD cause.
• Apply ESXi security patches on a fixed cadence. Do not wait for an outage.
• After any PSOD, capture the dump and the vm-support bundle before you patch or rebuild. Once you overwrite the host, the evidence is gone.

How Netdata helps

Netdata’s value for PSOD diagnosis is correlation, not prediction. PSODs themselves are sudden, but the precursors are usually visible in the hours or days before.

• Host connection state and HA failover capacity signals tell you immediately how bad the blast radius is and whether the cluster can absorb the restarts.
• VM heartbeat status lets you verify which VMs HA actually recovered versus which are still down.
• Hardware health sensors (correctable ECC rate, fan, PSU, temperature) collected via CIM providers surface the slow-burn memory or thermal failures that often precede a hardware PSOD.
• vmkernel.log error rate trending highlights the precursor warnings (MCE, path errors, driver asserts) that appear before the actual panic.
• Unexpected reboot detection, correlated with uptime counters and the absence of a planned maintenance event, gives you a clean signal that a crash happened even when nobody saw the screen.

Correlating these signals around the moment of the crash is what turns a vague “the host rebooted itself” ticket into a specific root cause.

Related guides

• vSphere active vs consumed vs granted memory: why the percentage lies
• vSphere CPU co-stop high (%CSTP): the SMP vCPU co-scheduling penalty
• vSphere CPU limit hit (%MLMTD): the forgotten MHz cap that silently throttles a VM
• vSphere CPU ready time high (%RDY): VMs starved while the guest looks idle
• vSphere datastore full: ‘No space left on device’, paused VMs, and power-on failures
• vSphere datastore IOPS and throughput: spotting storage saturation before latency bites
• vSphere datastore latency high: reading GAVG, DAVG, and KAVG
• vSphere dropped packets (%DRPRX/%DRPTX): ring buffers, CPU, and uplink backpressure
• vSphere storage latency cliff: the ’everything is slow’ incident that hits every VM at once
• vSphere host swapping (SWCUR/SWW/s): hypervisor swap and the memory death spiral
• How vSphere actually works in production: a mental model for operators
• vSphere memory ballooning (MCTLSZ): the host is reclaiming guest RAM