Source and classification

• Source channel: Engineering defects and a technical support case.
• Classification: Product issue.
• Confirmation basis: Engineering confirmed a bridge-name generation logic defect in ZSV-9030 and fixed subsequent host-expansion scenarios in ZSphere 4.10.12. ZSV-9054 further addressed host reconnection compatibility in historical environments with inconsistent bridge names.
• Internal tracking: ZSV-9030, ZSV-9054, and TIC-5338.

Issue metadata

• Issue ID: ZSV-9030.
• Component/product: ZSphere, distributed switches, and port groups.
• Status: Partially fixed. An upgrade does not automatically reconcile bridge names that are already inconsistent in a historical environment.
• Priority/severity: P0.
• Affected versions: Environments in which inconsistent bridge-name data was created on 4.10.11 or earlier and which are planned for upgrade to 4.10.12 or later.
• First fixed versions: 4.10.12 fixes bridge-name consistency when hosts are added after a VLAN change. 4.10.13 fixes host reconnection compatibility when historical bridge names are inconsistent.
• Issue definition date: 2025-06-06.
• Engineering confirmation date: 2025-06-09.
• Last updated: 2026-09-01.

Problem description

In versions earlier than 4.10.12, if a port group VLAN is changed in a cluster and a host is then added, the new host might generate a new bridge name based on the changed VLAN while existing hosts continue to use the old bridge name. The same port group therefore has inconsistent bridge names across hosts.

The expected behavior is for the same port group to use the same bridge name on every host in the cluster. ZSphere 4.10.12 changes the bridge-name generation logic so that subsequent host expansion preferentially reuses the port group's existing bridge-name record. However, the upgrade does not automatically change already-inconsistent host bridges or control-plane records in historical environments.

ZSphere 4.10.12 also adds a bridge-name consistency check. When a historical environment is upgraded directly to 4.10.12, hosts with pre-existing inconsistent bridge names might fail to reconnect. Through ZSV-9054, ZSphere 4.10.13 makes historical host reconnection compatible with this state, but it does not automatically reconcile existing inconsistent bridges. The cross-host migration risk must still be handled before the upgrade.

Symptoms

• After you run the check script on the management node, the database-internal resource identifier for the same port group (field name l2NetworkUuid) has multiple different bridgeName values. The script reports:

  bridgeName values are identical: no

• After a port group VLAN is changed and a host is added, the bridge names on the existing and new hosts differ. For example, an existing host uses br_dvs17_60 and a new host uses br_dvs17.
• A VM migration between hosts with different bridge names fails and might report:

  Cannot get interface MTU on 'br_dvs17_60': No such device

• After a historical environment is upgraded directly to 4.10.12, some hosts might fail to reconnect and report that the host's actual bridge name must match the bridge name recorded for the default switch.

Impact

• VMs that use an affected port group might not migrate between hosts with different bridge names.
• When the environment is upgraded directly to 4.10.12, hosts with historical bridge-name inconsistencies might fail to reconnect.
• Version 4.10.13 and later allow historical hosts to reconnect, but do not automatically correct inconsistencies between actual host bridges and control-plane records. Migration restrictions might remain.
• There is currently no evidence that this issue directly causes VM disk data loss.

Workaround

Pre-upgrade check

1. On the management node, create 4.10.12-upgrade-network-check.sh with the following content:

   #!/bin/bash

   # Database connection configuration
   DB_USER="root"
   DB_PASS="zstack.mysql.password"
   DB_NAME="zstack"

   if [ $# -ge 1 ]; then
       DB_PASS="$1"
       echo "Using the provided password"
   else
       echo "No password argument provided; using the default password"
   fi

   # Query the internal database resource IDs for all port groups
   l2NetworkUuids=$(mysql -u $DB_USER -p$DB_PASS $DB_NAME -N -e "SELECT DISTINCT l2NetworkUuid FROM L2NetworkHostRefVO;")

   # Check whether resource IDs were retrieved
   if [ -z "$l2NetworkUuids" ]; then
       echo "No l2NetworkUuid was found"
       exit 1
   fi

   # Iterate through each resource ID
   for uuid in $l2NetworkUuids; do
       echo "Processing l2NetworkUuid: $uuid"

       # Query the bridge names recorded for this resource ID on different hosts
       bridgeNames=$(mysql -u $DB_USER -p$DB_PASS $DB_NAME -N -e "SELECT bridgeName FROM L2NetworkHostRefVO WHERE l2NetworkUuid = '$uuid';")

       # Check whether bridge names were retrieved
       if [ -z "$bridgeNames" ]; then
           echo "No bridgeName was found"
           continue
       fi

       # Determine whether all bridge names are identical
       firstBridgeName=$(echo "$bridgeNames" | head -n 1)
       allSame=true

       for name in $bridgeNames; do
           if [ "$name" != "$firstBridgeName" ]; then
               allSame=false
               break
           fi
       done

       # Print the check result
       if $allSame; then
           echo "bridgeName values are identical: yes"
       else
           echo "bridgeName values are identical: no"
           echo "All information for this l2NetworkUuid:"
           mysql -u $DB_USER -p$DB_PASS $DB_NAME -e "SELECT * FROM L2NetworkHostRefVO WHERE l2NetworkUuid = '$uuid';"
       fi

       echo "----------------------------------------"
   done

2. Run the script on the management node:

   bash 4.10.12-upgrade-network-check.sh

If the database password at the site differs from the script's default value, pass the database root user password as the first argument:

   bash 4.10.12-upgrade-network-check.sh '<database-root-password>'

3. Check the script output:

• If all port group resource records report bridgeName values are identical: yes, continue the remaining pre-upgrade checks.
• If any port group resource record reports bridgeName values are identical: no, stop the upgrade and save the complete output.

Handling a detected inconsistency

1. Do not directly modify L2NetworkHostRefVO.bridgeName or SystemTagVO in the database, and do not directly rename, delete, or rebuild bridges on a host.
2. Collect the following information and contact engineering to develop a remediation plan for the current environment:

• Current version, planned target version, and upgrade path.
• Complete scan-script output.
• Affected clusters, distributed switches, port groups, database-internal resource identifiers for those port groups, and host UUIDs.
• Actual bridges, VLAN subinterfaces, and bridge link output from each relevant host.
• VMs using the port group and their current hosts.
• Operation times of the relevant VLAN change, host expansion, and migration failure.

3. During a maintenance window, stop or migrate away the VMs that use the affected port group. If cross-host migration has already failed, arrange VM downtime according to the engineering plan.
4. Engineering must provide a remediation script or procedure that matches the site's version and network state and reconcile these three types of state:

• Actual Linux bridges on the hosts.
• L2NetworkHostRefVO.bridgeName in the database.
• The port group's kvm::bridge::<bridgeName> bridge system tag.

5. After remediation, run the scan script again and verify that:

• bridgeName for the same l2NetworkUuid is consistent on all hosts.
• The relevant hosts reconnect normally.
• VMs that use the affected port group start normally.
• VMs can migrate in both directions between the relevant hosts.

6. Continue the upgrade only after completing these verifications.

There is currently no general, zero-downtime remediation for every historical environment. Determine the target bridge name from the existing system tags, the actual bridges on each host, the current VLAN, and VM distribution. Do not directly use the first bridge name in the scan results.

Affected products

• Product: ZSphere.
• Product line: ZVF.
• Components: Distributed switches, port groups, and host networking.
• Versions: Environments in which inconsistent bridge-name data was created on 4.10.11 or earlier and which are planned for upgrade to 4.10.12 or later.
• Trigger: A host is added after a port group VLAN is changed, and existing and new hosts record or create different bridge names for the same port group.

Fix status

• Current status: Partially fixed.
• Fixed versions:
• 4.10.12: ZSV-9030 fixes bridge-name consistency when hosts are added after a VLAN change.
• 4.10.13: ZSV-9054 allows hosts with historical bridge-name inconsistencies to reconnect after an upgrade.
• Fix limitation: These versions do not automatically reconcile inconsistent bridges that existed before the upgrade. If the pre-upgrade scan detects an inconsistency, engineering must still provide a targeted remediation plan.
• Related patch/fix package: No standalone general fix package is available.

Related information

• Related Jira: ZSV-9030.
• Upgrade reconnection compatibility Jira: ZSV-9054.
• Related technical support case: TIC-5338.
• Pre-upgrade check script: 4.10.12-upgrade-network-check.sh; its complete content is embedded in the Workaround section.