Issue/Introduction

This article provides a structured process for troubleshooting a black screen after opening a ZStack Cloud VM instance console. A black console does not necessarily mean that the VM instance failed to start. It might be caused by the console protocol, graphics type, guest operating system graphical interface, a post-migration driver, GPU passthrough, a Windows tool, or an abnormal program inside the operating system.

Symptoms

• The cloud platform console window opens, but the screen remains black or does not display a sign-in interface for a long time.
• The VM instance is accessible through SSH, Remote Desktop, or a workload port, but VNC or the console is black.
• After migration or import, a Linux or Windows VM instance starts with a black console.
• A GPU-passthrough VM instance or a VM instance on which GuestTools was installed restarts with a black screen.
• The screen remains black after the boot option is changed, virtio is disabled, or a snapshot is restored.

Environment

• Product: ZCF
• Component: Cloud
• Affected objects: ZStack Cloud; VM instance, console, VNC or SPICE, graphics type, GuestTools, and GPU passthrough
• Version: 4.x-4.8.38
• Deployment type: KVM VM instances; local storage, shared storage, or distributed storage might be involved

Possible causes

One or more of the following causes might apply:

• The console protocol or graphics type is incompatible. The VM instance console is black when SPICE is selected and recovers after switching to VNC. The compatibility of cirrus, qxl, and vga differs for GPU and graphics workloads.
• The guest operating system graphical interface is abnormal. A Linux desktop cannot start even though the command line or SSH works, or a Windows program crashes or security software consumes resources and delays console display.
• A post-migration or import compatibility issue exists. After migration from VMware or a V2V tool, virtio, CPU mode, graphics type, or the guest operating system version does not match.
• Windows GuestTools or a driver is abnormal. A GuestTools installation failure or abnormal virtio, QGA, or NIC driver causes the system to enter repair mode or display a black screen after restart.
• GPU passthrough or CPU mode does not match. In a GPU-passthrough scenario, the graphical interface cannot work and the CPU mode, graphics type, and guest driver must be checked.
• The guest operating system architecture is unsupported. In the applicable scenario, a Windows ARM VM instance is outside the current cloud platform compatibility boundary and displays a black console after creation.
• The console proxy or host network is abnormal. A console proxy port, host firewall, iptables, VNC port, or console proxy address is abnormal.

Resolution

Perform the following checks in order. At each step, confirm whether the current status meets expectations or identify the next action.

1. Determine whether only the console is black

• Check: Confirm whether the VM instance is still running and whether its workload is accessible.
• Command or UI path:
• On the cloud platform, check the VM instance status, host, IP address, recent tasks, and alerts.
• From the workload network, test SSH, RDP, ICMP, or the application port.
• On a Linux VM instance, you can try Ctrl+Alt+F2 to switch to a TTY.
• Expected result: You can distinguish a console display issue from a VM instance startup or operating system issue.
• If the result is abnormal:
• If workload access is normal and only the console is black, first troubleshoot the console protocol, graphics type, proxy, and guest graphical interface.
• If the workload is also inaccessible, troubleshoot VM instance startup, the operating system, storage, or the network.

2. Check the console protocol and graphics type

• Check: Confirm whether the console protocol is VNC and whether the graphics type is compatible with the guest operating system and workload.
• Command or UI path:
• Check the console type on the VM instance details page.
• Check the boot graphics type in the cluster advanced settings or VM instance configuration.
• Expected result: Prefer VNC for a standard scenario. Select the graphics type based on the guest operating system and graphics requirements.
• If the result is abnormal:
• If SPICE is selected and the console is black, change it to VNC, restart the VM instance, and verify the result.
• If cirrus causes a black screen for a GPU or graphical desktop, you can test qxl during a workload window.
• Changing the graphics type generally requires stopping or restarting the VM instance. Confirm the workload window first.

3. Check the guest operating system graphical interface and internal status

• Check: Confirm whether the black screen is caused by the guest operating system graphical interface, desktop service, system program, or security software.
• Command or UI path:
• Linux: Sign in through SSH or a TTY and check the graphical service, default boot target, and system logs.
• Windows: Sign in through RDP and check Event Viewer, resource usage, security software, and recently installed programs.
• Expected result: The guest operating system is accessible and has no persistent crashes, resource exhaustion, or abnormal desktop service.
• If the result is abnormal:
• If the Linux graphical interface is abnormal but the command line works, you can temporarily switch to command-line mode to verify whether the system starts normally.
• If many Windows programs crash, memory is exhausted, or security software consumes excessive resources, first resolve the program issue inside the guest operating system.
• If a system file is missing or software installation fails, create a snapshot or backup before repair.

4. Troubleshoot compatibility after migration or import

• Check: Confirm whether the VM instance was imported from VMware, V2V, or manually and whether the screen was black on the first startup after import.
• Command or UI path:
• Check the source operating system version, migration tool support boundary, volume controller, NIC model, CPU mode, and graphics type.
• Check whether virtio is enabled and whether GuestTools is installed on the VM instance.
• Expected result: The guest operating system, drivers, CPU mode, and graphics type are supported.
• If the result is abnormal:
• For a CentOS 8 VM instance migrated from VMware, you can test disabling virtio, setting the CPU mode to passthrough, or adjusting the graphics type.
• For a Linux VM instance migrated through V2V, if SSH works but the console is black, first confirm whether the graphical desktop is normal and then switch to command-line startup as needed.
• Before changing CPU mode, virtio, or the graphics type, preserve a snapshot and arrange a workload window.

5. Troubleshoot the Windows GuestTools and drivers

• Check: Confirm whether GuestTools was recently installed, upgraded, or reinstalled and whether Windows enters automatic repair or repeatedly displays a black screen.
• Command or UI path:
• On the cloud platform, check VM instance virtio, volume virtio, and the NIC type.
• In Windows Safe Mode or the recovery environment, check drivers, event logs, and the tool installation status.
• Expected result: Windows starts normally, and the virtio, QGA, and NIC drivers have a normal status.
• If the result is abnormal:
• In the applicable scenario, disabling VM instance virtio and volume virtio, changing the NIC type to e1000, and reinstalling GuestTools can resolve the issue.
• If the screen remains black after a snapshot is restored, confirm whether the snapshot already contains the abnormal driver or operating system corruption.
• Before repairing Windows drivers or GuestTools, it is recommended that you back up the root volume.

6. Troubleshoot GPU passthrough and graphics compatibility

• Check: Confirm whether a GPU, vGPU, A40, or another passthrough device is attached to the VM instance.
• Command or UI path:
• On the cloud platform, check the VM instance offering, passthrough devices, and boot graphics type.
• In the guest operating system, check the GPU driver and graphical service.
• Expected result: The GPU driver loads normally, and the CPU mode and graphics type are compatible with the guest operating system.
• If the result is abnormal:
• If an Ubuntu VM instance displays a black screen after GPU passthrough but can switch to a TTY, first treat it as a graphical interface or GPU driver issue.
• In the applicable scenario, setting the VM instance CPU mode to host-passthrough resolves the issue.
• For a black GPU VM instance console, you can test changing the boot graphics type from cirrus to qxl.

7. Check the console proxy, VNC port, and host firewall

• Check: Confirm whether the console proxy address, host VNC port, and iptables or firewalld are normal.
• Command or UI path:
• On the host where the VM instance resides, check whether the VNC port is listening.
• Check whether the host firewall or iptables allows the console-related ports.
• Compare whether the console recovers after migration to another host.
• Expected result: The console proxy is reachable, the VNC port exists, and firewall rules do not block it.
• If the result is abnormal:
• If the console works after migration to another host but becomes black after migration back to the original host, focus on the firewall, iptables, and console proxy on the original host.
• If the console proxy address is incorrect, check the configuration delivered by the management node and the host network.

8. Collect support information

• VM instance information: UUID, name, operating system type, source image, migration source, whether a GPU is attached, CPU mode, graphics type, and console protocol.
• Time window: First occurrence of the black screen and the most recent driver, tool, or patch installation, migration, and restart times.
• Management node logs: /usr/local/zstack/apache-tomcat/logs/management-server.log*
• Compute node logs: /var/log/zstack/zstack-kvmagent.log*
• Guest operating system evidence: Windows event logs, Linux journalctl, graphical service logs, and RDP or SSH availability.
• Host evidence: VNC listening ports, firewall rules, console proxy configuration, and the VM instance qemu process and logs.

Additional troubleshooting methods

Basic troubleshooting commands

# Management node services and basic cluster status
zstack-ctl status
systemctl status zstack-ha.service 2>/dev/null || true
systemctl status mariadb.service mysqld.service 2>/dev/null || true
zsha2 status 2>/dev/null || true

# Main management node log and recent errors
ls -lh /usr/local/zstack/apache-tomcat/logs/management-server.log*
grep -iE 'ERROR|WARN|Exception|failed|timeout' /usr/local/zstack/apache-tomcat/logs/management-server.log | tail -n 200

# Basic resources and system status
df -hT
free -m
top -b -n 1 | head -n 30
journalctl -p warning --since '2 hours ago' --no-pager | tail -n 200

Temporary workarounds

• First distinguish a management plane display issue, resource status issue, and underlying workload issue. Consider reconnecting, refreshing caches, retrying tasks, or restarting the management service only after confirming that the underlying status is normal.
• If the management node service must be restarted, first confirm the HA status, current tasks, and workload window. Prefer processing one node at a time and preserve /usr/local/zstack/apache-tomcat/logs/management-server.log*.
• Before repairing a database, forcibly deleting a resource, or reconnecting primary storage or image storage, complete a database backup and record a rollback plan.

VM instance and compute node checks

zstack-cli QueryVmInstance uuid=<vm-uuid>
zstack-cli QueryHost uuid=<host-uuid>
zstack-cli QueryVolume vmInstanceUuid=<vm-uuid>
virsh list --all
virsh domstate <vm-name-or-id>
virsh dumpxml <vm-name-or-id> > /tmp/vm.xml
systemctl status libvirtd zstack-kvmagent 2>/dev/null || true
grep -iE 'ERROR|Exception|failed|timeout|qmp|libvirt|domain' /var/log/zstack/zstack-kvmagent.log | tail -n 200
grep -iE '<vm-uuid>|<vm-name>' /usr/local/zstack/apache-tomcat/logs/management-server.log | tail -n 200

Temporary guidance for VM instance issues

• When a VM instance status is abnormal, compare the management plane, libvirt, QEMU process, and workload status before stopping or deleting based only on the UI status.
• For a black console, first check VNC or SPICE ports, the proxy, browser, host agent, and VM instance graphics configuration. If the workload is normal, you can temporarily use SSH or RDP sign-in instead.
• After a startup failure or abnormal shutdown, preserve the XML, agent logs, and management logs first. If necessary, migrate, reconnect, or refresh the status first. Before a forced operation, confirm volume and snapshot chain integrity.

Additional information

• Base the troubleshooting conclusion on the current version, service status, logs, cluster health, recent changes, and on-site impact.
• Before a high-risk operation such as a database change, forced deletion, OSD or MON rebuild, pool deletion, service restart, or network change, complete a backup, obtain approval, and obtain confirmation from ZStack Technical Support.