Issue/Introduction

After a VirtIO NIC is hot-added, the Windows PnP/netkvm device might become ready after the platform's first synchronization attempt. Two behaviors have been observed: Get-NetAdapter blocks beyond the host timeout, or the command returns quickly but cannot find the target MAC within the fixed retry window. The NIC becomes ready later, and the old task might continue or background synchronization might succeed.

Symptoms

• Network configuration immediately after Attach NIC fails in the first API call after about 120 seconds, or the log reports failed to find nic after retrying.
• QGA heartbeat is normal, but the target MAC does not appear in Get-NetAdapter during the first tens of seconds.
• The NIC appears after about one to several minutes, and a retry succeeds or the IP address eventually takes effect.
• The IP address eventually taking effect does not mean the first API call succeeded; an old guest process might continue after the timeout.

Environment

Verified for a hot-plugged Windows VirtIO NIC on ZSphere 5.1.0. Cold-start NICs, e1000, and MAC mismatches require separate analysis.

Possible causes

Correlate the timeline of hot-plug, initial configuration, Windows PnP device startup, the first QGA report of the MAC, and later success. A healthy QGA proves only that the channel is available, not that the NIC is ready.

Issue classification

• Type: Product bug/Windows PnP timing compatibility issue.
• Known issue: Yes. ZSV-12857 was Open/P1 as of 2026-08-06, affects version 5.1.0, and has no confirmed fixed version.
• Root cause: Initial configuration runs before the VirtIO/netkvm device is ready. The fixed wait and host timeout do not cover Windows PnP readiness time, and the guest process might not terminate after timeout.
• Confidence: High, provided that the complete timeline is consistent.

Resolution

Check 1: Confirm whether the target MAC has been enumerated

• Read-only PowerShell:

  Get-NetAdapter -IncludeHidden |
    Format-Table Name,ifIndex,Status,MacAddress,InterfaceDescription -Auto
  Get-PnpDevice -PresentOnly | Where-Object { $_.FriendlyName -match 'VirtIO|Ethernet' } |
    Format-Table Status,Class,FriendlyName,InstanceId -Auto

• If the target MAC is absent immediately after the platform completed the hot-plug: This supports the PnP-not-ready branch. Continue to check event times.
• If the target MAC exists and its state is Up: The initial failure might be stale. Still inspect the zs-tools log; do not immediately declare the issue resolved.
• If the MAC differs from the platform record: Troubleshoot MAC/inventory instead; this article does not apply.

Check 2: Correlate Windows PnP events

• Read-only PowerShell:

  Get-WinEvent -LogName 'Microsoft-Windows-Kernel-PnP/Configuration' -MaxEvents 100 |
    Where-Object { $_.Id -in 400,410 } |
    Select-Object TimeCreated,Id,LevelDisplayName,Message
  Get-WinEvent -LogName System -MaxEvents 200 |
    Where-Object { $_.Id -eq 7036 -and $_.Message -match 'Network Setup' } |
    Select-Object TimeCreated,Id,Message

• If device startup completed after the first configuration failure: The timing evidence is established.
• If PnP completed much earlier: Check driver errors, MAC matching, or the guest tool itself.

Check 3: Correlate platform and GuestTools logs

• Read-only search: Use the VM UUID/MAC/API time to search the audit log, zstack-kvmagent.log, and in-guest zs-tools.log.
• If the guest log continues after the 120-second host timeout, or duplicate commands appear: There is a residual-process/concurrency risk.
• If all five fixed retries fail to find the MAC and QGA reports the same MAC for the first time later: This matches another manifestation of ZSV-12857.

Recommended actions

After hot-add, wait until the target MAC appears in Windows and the device status is normal, and then manually retry network configuration once. Before retrying, confirm that no old zs-tools net task is still running to avoid concurrent network changes. Production network changes require console/out-of-band access and a rollback plan for the original configuration. Do not retry repeatedly just because QGA is online.

Long-term solution

Track ZSV-12857: wait for device readiness by target MAC; set deadlines for subcommands and the complete process tree; terminate the guest task after a host timeout; and add a global lock plus PID/duration/last-subcommand output to zs-tools net.

Additional information

When to escalate

• The target MAC does not appear for a long time or the device shows a yellow warning icon.
• Multiple residual zs-tools processes remain after configuration fails.
• Configuration still fails after waiting for the device to become ready.
• A process must be terminated, a service/guest restarted, or a production IP changed.

Information to collect

ZSphere/Windows/VirtIO driver versions, VM and NIC UUID/MAC, API and hot-plug timestamps, audit records, KVM Agent logs, zs-tools logs, PnP 400/410 events, and System 7036 events.

Related issues

• ZSV-12857, Open/P1.
• Status verified on 2026-08-06.