Issue/Introduction

This article applies when a UI details page loads slowly, a resource list and details page display inconsistent data, monitoring data is inaccurate, alert acknowledgment status is abnormal, an audit page is blank, a permission button is not visible, or role permissions are abnormal. Distinguish the frontend display, backend API, monitoring metric definition, permission model, browser cache, and actual workload status.

Symptoms

• A VM instance details page loads slowly or displays no sign-in interface.
• TOP metrics on the home page differ from monitoring metrics on the details page.
• The number of alarm messages, acknowledgment status, event details, or email template content does not meet expectations.
• The audit page is blank for the admin user, and the browser developer tools display abnormal frontend data.
• Project roles, permission changes, or load balancer backend server permissions are displayed incorrectly.
• Exporting VM instance information, tag management, DNS, CDP details, or other UI content is displayed incorrectly.

Environment

• Product: ZCF
• Component: Cloud
• Affected objects: ZStack Cloud; UI, monitoring, alert, permissions, audit, VM instance details page, API, and browser
• Version: 4.x-4.8.38

Possible causes

• The home page and details page use different metricName values, aggregation definitions, or time windows.
• GuestTools on the VM instance, the internal monitoring forwarding service, or a related component is abnormal, causing the UI to wait for data.
• A frontend chunk, pagination, cache, browser state, or an undefined API response causes a blank page.
• Platform roles, project roles, child API permissions, or resource visibility differ in the permission model.
• Alert acknowledgment, event details, template tags, or alert count logic differs from user expectations.
• A UI bug or known issue requires an upgrade or patch.

Resolution

1. Distinguish a display issue from the actual status

• Cross-check the same resource on the list page, details page, CLI or API, database, and underlying commands.
• Retest in a private browser window, after clearing the cache, in another browser, or with another account to exclude a local cache issue.

2. Check the frontend and API

• Press F12 to open the browser developer tools and inspect Console, Network, API responses, and failed requests.
• Use API Inspector or management logs to confirm whether the backend returns data normally.
• If the API response is normal but the UI display is abnormal, first record it as a frontend issue.

3. Check the monitoring path

ps -ef | grep light
ip netns list
grep -a -n "<vmUuid-or-metricName>" /usr/local/zstack/apache-tomcat/logs/management-server.log*

• If internal monitoring has no data, check GuestTools, DHCP or userdata, forwarding service, and metric collection path.
• If metrics differ, first confirm whether different metrics or aggregation definitions are used.

4. Check permissions and roles

• Record the user, role, project, resource, button or API, and expected permission.
• Check whether the permission is available only to a platform role or whether a child API is not authorized.
• For a permission issue, it is recommended that you retest with a minimal reproduction account and role configuration.

Additional troubleshooting methods

Basic troubleshooting commands

# Management node services and basic cluster status
zstack-ctl status
systemctl status zstack-ha.service 2>/dev/null || true
systemctl status mariadb.service mysqld.service 2>/dev/null || true
zsha2 status 2>/dev/null || true

# Main management node log and recent errors
ls -lh /usr/local/zstack/apache-tomcat/logs/management-server.log*
grep -iE 'ERROR|WARN|Exception|failed|timeout' /usr/local/zstack/apache-tomcat/logs/management-server.log | tail -n 200

# Basic resources and system status
df -hT
free -m
top -b -n 1 | head -n 30
journalctl -p warning --since '2 hours ago' --no-pager | tail -n 200

Temporary workarounds

• First distinguish a management plane display issue, resource status issue, and underlying workload issue. Consider reconnecting, refreshing caches, retrying tasks, or restarting the management service only after confirming that the underlying status is normal.
• If the management node service must be restarted, first confirm the HA status, current tasks, and workload window. Prefer processing one node at a time and preserve /usr/local/zstack/apache-tomcat/logs/management-server.log*.
• Before repairing a database, forcibly deleting a resource, or reconnecting primary storage or image storage, complete a database backup and record a rollback plan.

VM instance and compute node checks

zstack-cli QueryVmInstance uuid=<vm-uuid>
zstack-cli QueryHost uuid=<host-uuid>
zstack-cli QueryVolume vmInstanceUuid=<vm-uuid>
virsh list --all
virsh domstate <vm-name-or-id>
virsh dumpxml <vm-name-or-id> > /tmp/vm.xml
systemctl status libvirtd zstack-kvmagent 2>/dev/null || true
grep -iE 'ERROR|Exception|failed|timeout|qmp|libvirt|domain' /var/log/zstack/zstack-kvmagent.log | tail -n 200
grep -iE '<vm-uuid>|<vm-name>' /usr/local/zstack/apache-tomcat/logs/management-server.log | tail -n 200

Temporary guidance for VM instance issues

• When a VM instance status is abnormal, compare the management plane, libvirt, QEMU process, and workload status before stopping or deleting based only on the UI status.
• For a black console, first check VNC or SPICE ports, the proxy, browser, host agent, and VM instance graphics configuration. If the workload is normal, you can temporarily use SSH or RDP sign-in instead.
• After a startup failure or abnormal shutdown, preserve the XML, agent logs, and management logs first. If necessary, migrate, reconnect, or refresh the status first. Before a forced operation, confirm volume and snapshot chain integrity.

UI, account, permission, and license checks

# Management service, time, and frontend access
zstack-ctl status
date
chronyc sources -v 2>/dev/null || ntpq -p 2>/dev/null || true
curl -k -I https://<management-ip>:5000/ 2>/dev/null || curl -I http://<management-ip>:8080/

# Management log keywords
grep -iE 'login|token|license|permission|account|project|billing|invoice|ERROR|Exception' /usr/local/zstack/apache-tomcat/logs/management-server.log | tail -n 200

Temporary guidance for UI and permission issues

• For a sign-in or page issue, first exclude browser cache, client time drift, management node time synchronization, and reverse proxy or certificate issues.
• For an account, project, permission, billing, or invoice issue, first record the account UUID, project UUID, operation path, and API response. Do not directly change database permission or billing records.
• If license upload fails, first check the product version, license file validity period, machine code, and management node time. You can temporarily restore the original license or pause the upgrade change window.

Additional information

• Base the troubleshooting conclusion on the current version, service status, logs, cluster health, recent changes, and on-site impact.
• Before a high-risk operation such as a database change, forced deletion, OSD or MON rebuild, pool deletion, service restart, or network change, complete a backup, obtain approval, and obtain confirmation from ZStack Technical Support.