Overview

To upgrade ZStack Cloud, you need to upgrade the management nodes, compute nodes, storage nodes, and network nodes to the latest version and update basic software packages, software library dependencies, and core virtualization tools. This document describes how to upgrade ZStack Cloud in details, including:

Limitations

This document applies to cloud platform upgrading of the same repository (repo), which means:
  • The management nodes and all hosts on the cloud platform are using the same operating system (OS) version, for example, all use the h84r OS.
  • The OS version used by the target ISO is consistent with OS version currently used by the management nodes and hosts. For example, if all management nodes and hosts on the cloud platform are using the h84r OS, you must use a ZStack Cloud ISO of the h84r edition to perform the upgrading.

Upgrading Mechanism

ZStack Cloud provides the following upgrading mechanisms:
  • Seamless upgrading: In a same-repo environment, ZStack Cloud of any old version can be upgraded to the latest version.
  • Lossless upgrading: The upgrade process does not affect the services running in VM instances.
Note:
  • This document takes upgrading the cloud platform of same-repo with the h84r ISO as an example.
  • If you need to change host OS or upgrade a cloud platform of multiple repos, contact the official technical support for help.

Considerations

Before upgrading ZStack Cloud, be aware of the following points:
  • During the upgrading, the management nodes are inaccessible and you cannot apply VM and volume resources. Therefore, we recommend that you upgrade the cloud platform during business troughs to avoid resource application and business change needs.
  • If you upgrade the virtualization tools (QEMU & Libvirt), it is not supported to rollback the tools to earlier versions.
  • If you upgrade the host QEMU, you need to reboot or migrate VM instances, VPC vRouters, and LB instances on the hosts to make the new QEMU version take effect.
  • If you upgrade the host Kernel, you need to restart the host to make the new Kernel version take effect.
  • If any errors occur during the upgrading, interrupting the upgrading process or affecting the running business, contact the official technical support immediately.
  • Read the feature update information to learn about the key feature changes in the target version in advance.
Version Upgrade Information Details
ZStack Cloud 5.1.0 Removes the UI button Set VM RDP Mode. Starting from 5.1.0, it is not supported to set the VM RDP mode on the UI. To enable the RDP mode for a VM instance, run the CLI command SetVmRDP uuid=${vmuuid} enable=true
ZStack Cloud 4.7.21 Renames Entertainment Management as Tenant Management. Starting from 4.7.21, the feature module Entertainment Management is renamed as Tenant Management. This does not affect the functions of this module.
ZStack Cloud 4.6.0 Upgrades Affinity Group to VM Scheduling Policy. Starting from 4.6.0, the feature Affinity Group is upgraded to VM Scheduling Policy:
  • The VM Scheduling Policy feature provides four types of policies: VM Exclusive from Each Other, VM Affinitive to Each Other, VMs Affinitive to Hosts, and VMs Exclusive from Hosts. Each policy has two execution mechanisms: Hard and Soft.
  • Anti-Affinity Group (Hard) and Anti-Affinity Group (Soft) policies provided in earlier versions are upgraded to VM Affinitive to Each Other (Hard) and VM Affinitive to Each Other (Soft) respectively, without any modification to the underlying working principles. This upgrade has no impact on your existing application workloads and is annoyance-free.
ZStack Cloud 4.0.0 Upgrades vRouter to VPC vRouter. Starting from 4.0.0, vRouter is upgraded to VPC vRouter and vRouter network is upgraded to VPC network. vRouter is no longer available on the UI. The seamless upgrade does not affect your business continuity.
Replaces User Group and User by the account system in Entertainment Management. Starting from 4.0.0, User Group and User are replaced by the system account in Entertainment Management Account System. User Group and User are no longer available on the UI. You cannot login to the Cloud by using a user or user group account. Before you implement upgrading, migrate the user and user group account data to Entertainment Management.
Note: User accounts created by the admin and have the admin permissions are also can celled. Platform admins in Entertainment Management have the equivalent permissions. You can use Entertainment Management to create platform managers based on your business needs.
Adjusts the integration of AD/LDAP and accounts. Starting from 4.0.0, the integration of AD/LDAP and accounts is managed by Entertainment Management. Before you implement upgrading, migrate the AD/LDAP account data managed by the original account system to Entertainment Management.

Upgrade Cloud Platform

Manually upgrade the management nodes of ZStack Cloud to the latest version. Then, automatically or manually upgrades resources such as hosts, primary storage, image storage, and VPC vRouters, depending on the selected upgrade method.

ZStack Cloud supports two platform upgrade methods: standard upgrade and grayscale upgrade, for both single-management node and dual-management node deployments. Upgrade procedures vary by upgrade method and deployment method. Select the appropriate upgrade method and procedure based on the actual deployment scenario:
  • Standard Upgrade

    Standard upgrade method upgrades the management node first. Then, the platform automatically reconnects and upgrades resources such as hosts, primary storage, image storage, and VPC vRouters in a single operation. This method does not support batch-based or phased upgrade control.

  • Grayscale Upgrade

    Grayscale upgrade method upgrades the management node first but does not immediately upgrade all hosts and other nodes. The platform advanc

    es the upgrade in batches by using grayscale control. You can prioritize hosts that run noncritical workloads for validation. After verification completes without issues, gradually proceed to a full upgrade to reduce upgrade risk.

    Note:

    The grayscale upgrade feature is available only in ZStack Cloud 5.0.0 and later. Earlier versions do not support this upgrade mode.

Single-Management Node Standard Upgrading

About this task

This topic describes the standard upgrading process of a single-management node platform.

Procedure

  1. Obtain software packages for upgrading.
    Prepare the following packages and save them in the /root/ directory of the management node:
    • ZStack Cloud ISO:
      • Software: ZStack-Cloud-x86_64-DVD-5.5.24-h84r.iso
      • Download address: Click here
    • ZStack Cloud installation package:
      • Software: ZStack-Cloud-installer-5.5.24.bin
      • Download address: Click here
    • ZStack Cloud Upgrade Script:
      • Software: ZStack-Cloud-upgrade
      • Download address: Click here
    Note:
    • ZStack-Cloud supports offline upgrading, which allows you to upgrade the cloud platform without connecting to the public network. Therefore, you need to download the software packages and upgrade script in advance.
    • After you download the software packages and upgrade script, check the MD5 hash of the packages or scripts. If the MD5 hash is inconsistent with that on the official website, download again.
  2. Disable VM HA globally.

    Login to the UI, choose Settings > Platform Setting > HA Policy, on the HA Policy page, click the Disable button on the upper right corner.

  3. Back up the database and upgrading script.
    Before you upgrade the cloud platform, back up the database and upgrading script, in case of any unexpected situations.
    • Run the following command on the management node to back up the database in the /var/lib/zstack/mysql-backup/ directory:
      [root@localhost ~]# zstack-ctl dump_mysql --file-name zstack-db-backup
    • Run the following command on the management node to back up the upgrading script in the /root/ directory:
      [root@localhost ~]# cp /usr/local/bin/zstack-upgrade /root/zstack-upgrade-bk
  4. Upgrade the cloud platform.
    Run the following commands on the management node to implement offline upgrading:
    # Two offline upgrading methods are supported. The first one is recommended:
    # First method: Upgrade the local repo and management service simultaneously:
    [root@zstack-1 opt]# zstack-upgrade ZStack-Cloud-x86_64-DVD-5.5.24-h84r.iso
    # Second method: Upgrade the local repo first and then the management service:
    [root@localhost ~]# zstack-upgrade -r ZStack-Cloud-x86_64-DVD-5.5.24-h84r.iso
    [root@localhost ~]# bash ZStack-Cloud-installer-5.5.24.bin -u -P <MYSQL_ROOT_PASSWORD>
    Note:
    • <MYSQL_ROOT_PASSWORD> is the password of the MySQL root user. The placeholder is for example purposes only and must be replaced with the actual password.
    • If the root password of the database is the default password, you do not need to specify -P <MYSQL_ROOT_PASSWORD>.
    • Before you run the zstack-upgrade command, make sure that you use ISO of the h84r version instead of other versions.
    • While upgrading the local repo, the upgrade script is auto-updated. When the update is completed, rerun the preceding command.
  5. Clean up the browser cache.

    To make sure that the upgraded platform works as expected, after the upgrading is completed, you need to login to the UI platform and press Ctrl+F5 or Ctrl+Shift+R to clean up the browser cache.

  6. Enable VM HA globally.

    Login to the UI, choose Settings > Platform Setting > HA Policy, on the HA Policy page, click the Enable button on the upper right corner.

What to do next

  • Now, the management node has been upgraded. The system will automatically reconnects other nodes, such as hosts, primary storage, image storage, and VPC vRouters, to complete the cloud platform upgrading.
  • You can continue to upgrade virtualization tools and Kernel as needed.

Dual-Management Node Standard Upgrading

About this task

This topic describes the standard upgrading process of a dual-management node platform.

Procedure

  1. Obtain software packages for upgrading.
    Prepare the following packages and save them respectively in the /root/ directory of the two management nodes:
    • ZStack Cloud ISO:
      • Software: ZStack-Cloud-x86_64-DVD-5.5.24-h84r.iso
      • Download address: Click here
    • ZStack Cloud installation package:
      • Software: ZStack-Cloud-installer-5.5.24.bin
      • Download address: Click here
    • ZStack Cloud Upgrade Script:
      • Software: ZStack-Cloud-upgrade
      • Download address: Click here
    • Multi-MN-HA Suite:
      • Software: ZStack-Multinode-HA-Suite-5.5.24.tar.gz
      • Download address: Click here
    Note:
    • ZStack-Cloud supports offline upgrading, which allows you to upgrade the cloud platform without connecting to the public network. Therefore, you need to download the software packages and upgrade script in advance.
    • After you download the software packages and upgrade script, check the MD5 hash of the packages or scripts. If the MD5 hash is inconsistent with that on the official website, download again.
  2. Disable VM HA globally.

    Login to the UI, choose Settings > Platform Setting > HA Policy, on the HA Policy page, click the Disable button on the upper right corner.

  3. Back up the database and upgrading script.
    Before you upgrade the cloud platform, back up the database and upgrading script, in case of any unexpected situations.
    • Run the following command on the management nodes respectively to back up the database in the /var/lib/zstack/mysql-backup/ directory:
      [root@localhost ~]# zstack-ctl dump_mysql --file-name zstack-db-backup
    • Run the following command on the management nodes respectively to back up the upgrading script in the /root/ directory:
      [root@localhost ~]# cp /usr/local/bin/zstack-upgrade /root/zstack-upgrade-bk
  4. Update the local repo.
    Run the following command on the management nodes respectively to update the local repo:
    [root@localhost ~]# bash zstack-upgrade -r ZStack-Cloud-x86_64-DVD-5.5.24-h84r.iso
    Note: While upgrading the local repo, the upgrade script is auto-updated. When the update is completed, rerun the preceding command.
  5. Obtain the IP address of the management node that corresponds to the VIP you use.
    Run the following command to obtain the IP address of the management node that corresponds to the VIP you use. The management node that corresponds to the yes result is the management node you need:
    [root@localhost ~]# zsha2 status
  6. Decompress the Multi-MN-HA file and grant the decompressed files execution permission.
    Run the following commands on the management node that corresponds to the VIP you use to decompress the Multi-MN-HA file and grant the decompressed files zsha2 and zstack-hamon execution permission:
    [root@localhost ~]# tar zxvf ZStack-Multinode-HA-Suite-5.5.24.tar.gz      #Decompress the Multi-MN-HA file
    [root@localhost ~]# chmod +x zsha2 zstack-hamon    #Grant the decompressed files execution permission
  7. Upgrade Multi-MN-HA Suite
    Run the following command on the management node that corresponds to the VIP you use to upgrade Multi-MN-HA Suite:
    [root@localhost ~]# ./zsha2 upgrade-ha
  8. Upgrade the cloud platform.
    Run the following command on the management node that corresponds to the VIP you use to upgrade the management nodes:
    zsha2 upgrade-mn -peerpass password ZStack-Cloud-installer-5.5.24.bin
    Note: -peerpass is optional. You can use the parameter to set the password of SSH login to the peer management nodes.
  9. Clean up the browser cache.

    To make sure that the upgraded platform works as expected, after the upgrading is completed, you need to login to the UI platform and press Ctrl+F5 or Ctrl+Shift+R to clean up the browser cache.

  10. Enable VM HA globally.

    Login to the UI, choose Settings > Platform Setting > HA Policy, on the HA Policy page, click the Enable button on the upper right corner.

What to do next

  • Now, the management nodes have been upgraded. The system will automatically reconnects other nodes, such as hosts, primary storage, image storage, and VPC vRouters, to complete the cloud platform upgrading.
  • You can continue to upgrade virtualization tools and Kernel as needed.

Single-Management Node Grayscale Upgrading

About this task

This topic describes the grayscale upgrading process of a single-management node platform.

Procedure

  1. Obtain software packages for upgrading.
    Prepare the following packages and save them in the /root/ directory of the management node:
    • ZStack Cloud ISO:
      • Software: ZStack-Cloud-x86_64-DVD-5.5.24-h84r.iso
      • Download address: Click here
    • ZStack Cloud installation package:
      • Software: ZStack-Cloud-installer-5.5.24.bin
      • Download address: Click here
    • ZStack Cloud Upgrade Script:
      • Software: ZStack-Cloud-upgrade
      • Download address: Click here
    Note:
    • ZStack-Cloud supports offline upgrading, which allows you to upgrade the cloud platform without connecting to the public network. Therefore, you need to download the software packages and upgrade script in advance.
    • After you download the software packages and upgrade script, check the MD5 hash of the packages or scripts. If the MD5 hash is inconsistent with that on the official website, download again.
  2. Disable VM HA globally.

    Login to the UI, choose Settings > Platform Setting > HA Policy, on the HA Policy page, click the Disable button on the upper right corner.

  3. Select a host for grayscale upgrade validation.

    During the pre-upgrade preparation phase, select a host that runs noncritical workloads as the grayscale upgrade validation node. This host will initiate manual reconnection first in the following upgrade process to verify the agent upgrade, VM instance runtime status, and service availability.

  4. Back up the database and upgrading script.
    Before you upgrade the cloud platform, back up the database and upgrading script, in case of any unexpected situations.
    • Run the following command on the management node to back up the database in the /var/lib/zstack/mysql-backup/ directory:
      [root@localhost ~]# zstack-ctl dump_mysql --file-name zstack-db-backup
    • Run the following command on the management node to back up the upgrading script in the /root/ directory:
      [root@localhost ~]# cp /usr/local/bin/zstack-upgrade /root/zstack-upgrade-bk
  5. Grayscale upgrade the cloud platform.
    Run the following commands on the management node to implement grayscale upgrading:
    [root@localhost ~]# zstack-upgrade -r ZStack-Cloud-x86_64-DVD-5.5.24-h84r.iso      # Update the local repo.
    [root@localhost ~]# bash zstack-installer-5.5.24.bin -u --grayscale true -P <MYSQL_ROOT_PASSWORD>      # Grayscale upgrade the cloud platform.
    Note:
    • <MYSQL_ROOT_PASSWORD> is the password of the MySQL root user. The placeholder is for example purposes only and must be replaced with the actual password.
    • If the root password of the database is the default password, you do not need to specify -P <MYSQL_ROOT_PASSWORD>.
    • -grayscale enables grayscale upgrade mode. After the upgrade completes, the platform does not automatically upgrade Hosts or related Agents.
    • After the upgrade completes, resources such as hosts and VPCs remain as Not Upgraded. Proceed with the upgrade by reconnecting the resources or disabling grayscale mode.
  6. Clean up the browser cache.

    To make sure that the upgraded platform works as expected, after the upgrading is completed, you need to login to the UI platform and press Ctrl+F5 or Ctrl+Shift+R to clean up the browser cache.

  7. Check the resource status after graysacle upgrade.

    Log in to the ZStack Cloud UI and check the status of hosts, primary storage, image storage and other resources. Ensure that each resource is in the Connected state. Besides, check the Upgrade Status of resources such as hosts and VPCs. Under the grayscale upgrade mode, the Upgrade Status should be displayed as Not Upgraded at this time.

  8. Perform grayscale upgrade validation on a noncritical host.

    In the ZStack Cloud UI, manually reconnect the predesigned host that runs noncritical workloads to trigger upgrades for the host and related agents.

    After the reconnection completes, verify the upgrade results to ensure that the upgrade process completes successfully and that services remain operational:
    • Verify that the agent version on the host is updated and the host runs normally.
    • Check that the primary storage and related components operate normally.
    • Verify that VM instances run normally, network connectivity works as expected, and service access remains unaffected.

    When all verification checks pass and the host upgrade status displays Upgraded, the grayscale upgrade validation completes successfully. The subsequent full-platform upgrade can then proceed.

    Note:
    • During a grayscale upgrade, you can reconnect hosts that have not been upgraded. Reconnection automatically triggers the agent upgrade.
    • If a host disconnects unexpectedly during the grayscale upgrade and automatic reconnection fails, manually reconnect the host to restore connectivity. The manual reconnection also upgrades the agent.
  9. Proceed with full-platform resource upgrade.

    After the grayscale upgrade validation completes, use one of the following methods to upgrade the remaining resources based on operational requirements:

    • Manual phased upgrade: Reconnect the remaining hosts one by one to gradually upgrade the hosts and related agents. Continuously monitor the upgrade progress and service status.
    • Automatic upgrade: After the grayscale upgrade validation on noncritical hosts completes and no further phased validation is required, disable grayscale mode. The platform automatically upgrades the remaining hosts and related resources.

    Regardless of the method you choose, continuously monitor VM instance status during the upgrade to ensure that services remain unaffected.

  10. Disable Grayscale Upgrade Mode

    After you confirm that the grayscale upgrade validation completes without issues, run the following command to disable the grayscale upgrade mode:

    [root@localhost ~]# zstack-cli   // Enter zstack-cli.
    admin >>> LogInByAccount accountName=admin password=<ADMIN_PASSWORD>     // Log in to the CLI with the admin account. <ADMIN_PASSWORD> is the password of the admin account and is used as a placeholder here.
    admin >>> UpdateGlobalConfig name=grayscaleUpgrade category=upgradeControl value=false     // Disable the grayscale mode.
  11. Enable VM HA globally.

    Login to the UI, choose Settings > Platform Setting > HA Policy, on the HA Policy page, click the Enable button on the upper right corner.

What to do next

Now, the platform grayscale upgrade is complete in a single-management node deployment. You can continue to upgrade virtualization tools and Kernel as needed.

Dual-Management Node Grayscale Upgrading

About this task

This topic describes the grayscale upgrading process of a dual-management node platform.

Procedure

  1. Obtain software packages for upgrading.
    Prepare the following packages and save them respectively in the /root/ directory of the two management nodes:
    • ZStack Cloud ISO:
      • Software: ZStack-Cloud-x86_64-DVD-5.5.24-h84r.iso
      • Download address: Click here
    • ZStack Cloud installation package:
      • Software: ZStack-Cloud-installer-5.5.24.bin
      • Download address: Click here
    • ZStack Cloud Upgrade Script:
      • Software: ZStack-Cloud-upgrade
      • Download address: Click here
    • Multi-MN-HA Suite:
      • Software: ZStack-Multinode-HA-Suite-5.5.24.tar.gz
      • Download address: Click here
    Note:
    • ZStack-Cloud supports offline upgrading, which allows you to upgrade the cloud platform without connecting to the public network. Therefore, you need to download the software packages and upgrade script in advance.
    • After you download the software packages and upgrade script, check the MD5 hash of the packages or scripts. If the MD5 hash is inconsistent with that on the official website, download again.
  2. Disable VM HA globally.

    Login to the UI, choose Settings > Platform Setting > HA Policy, on the HA Policy page, click the Disable button on the upper right corner.

  3. Select a host for grayscale upgrade validation.

    During the pre-upgrade preparation phase, select a host that runs noncritical workloads as the grayscale upgrade validation node. This host will initiate manual reconnection first in the following upgrade process to verify the agent upgrade, VM instance runtime status, and service availability.

  4. Back up the database and upgrading script.
    Before you upgrade the cloud platform, back up the database and upgrading script, in case of any unexpected situations.
    • Run the following command on the management nodes respectively to back up the database in the /var/lib/zstack/mysql-backup/ directory:
      [root@localhost ~]# zstack-ctl dump_mysql --file-name zstack-db-backup
    • Run the following command on the management nodes respectively to back up the upgrading script in the /root/ directory:
      [root@localhost ~]# cp /usr/local/bin/zstack-upgrade /root/zstack-upgrade-bk
  5. Update the local repo.
    Run the following command on the management nodes respectively to update the local repo:
    [root@localhost ~]# bash zstack-upgrade -r ZStack-Cloud-x86_64-DVD-5.5.24-h84r.iso
    Note: While upgrading the local repo, the upgrade script is auto-updated. When the update is completed, rerun the preceding command.
  6. Obtain the IP address of the management node that corresponds to the VIP you use.
    Run the following command to obtain the IP address of the management node that corresponds to the VIP you use. The management node that corresponds to the yes result is the management node you need:
    [root@localhost ~]# zsha2 status
  7. Decompress the Multi-MN-HA file and grant the decompressed files execution permission.
    Run the following commands on the management node that corresponds to the VIP you use to decompress the Multi-MN-HA file and grant the decompressed files zsha2 and zstack-hamon execution permission:
    [root@localhost ~]# tar zxvf ZStack-Multinode-HA-Suite-5.5.24.tar.gz      #Decompress the Multi-MN-HA file
    [root@localhost ~]# chmod +x zsha2 zstack-hamon    #Grant the decompressed files execution permission
  8. Grayscale Upgrade Multi-MN-HA Suite
    Run the following command on the management node that corresponds to the VIP you use to grayscale upgrade Multi-MN-HA Suite:
    [root@localhost ~]# ./zsha2 upgrade-ha -gencfg=true
  9. Grayscale upgrade the cloud platform.
    Run the following command on the management node that corresponds to the VIP you use to grayscale upgrade the management nodes:
    [root@localhost ~]# ./zsha2 upgrade-mn -grayscale -yes -db-root-pw <MYSQL_ROOT_PASSWORD> -peerpass password ZStack-Cloud-installer-5.5.24.bin
    Note:
    • <MYSQL_ROOT_PASSWORD> is the password of the MySQL root user. The placeholder is for example purposes only and must be replaced with the actual password.
    • -grayscale enables grayscale upgrade mode. After the upgrade completes, the platform does not automatically upgrade Hosts or related Agents.
    • -peerpass is optional. You can use the parameter to set the password of SSH login to the peer management nodes.
    • After the upgrade completes, resources such as hosts and VPCs remain as Not Upgraded. Proceed with the upgrade by reconnecting the resources or disabling grayscale mode.
  10. Clean up the browser cache.

    To make sure that the upgraded platform works as expected, after the upgrading is completed, you need to login to the UI platform and press Ctrl+F5 or Ctrl+Shift+R to clean up the browser cache.

  11. Check the resource status after graysacle upgrade.

    Log in to the ZStack Cloud UI and check the status of hosts, primary storage, image storage and other resources. Ensure that each resource is in the Connected state. Besides, check the Upgrade Status of resources such as hosts and VPCs. Under the grayscale upgrade mode, the Upgrade Status should be displayed as Not Upgraded at this time.

  12. Perform grayscale upgrade validation on a noncritical host.

    In the ZStack Cloud UI, manually reconnect the predesigned host that runs noncritical workloads to trigger upgrades for the host and related agents.

    After the reconnection completes, verify the upgrade results to ensure that the upgrade process completes successfully and that services remain operational:
    • Verify that the agent version on the host is updated and the host runs normally.
    • Check that the primary storage and related components operate normally.
    • Verify that VM instances run normally, network connectivity works as expected, and service access remains unaffected.

    When all verification checks pass and the host upgrade status displays Upgraded, the grayscale upgrade validation completes successfully. The subsequent full-platform upgrade can then proceed.

    Note:
    • During a grayscale upgrade, you can reconnect hosts that have not been upgraded. Reconnection automatically triggers the agent upgrade.
    • If a host disconnects unexpectedly during the grayscale upgrade and automatic reconnection fails, manually reconnect the host to restore connectivity. The manual reconnection also upgrades the agent.
  13. Proceed with full-platform resource upgrade.

    After the grayscale upgrade validation completes, use one of the following methods to upgrade the remaining resources based on operational requirements:

    • Manual phased upgrade: Reconnect the remaining hosts one by one to gradually upgrade the hosts and related agents. Continuously monitor the upgrade progress and service status.
    • Automatic upgrade: After the grayscale upgrade validation on noncritical hosts completes and no further phased validation is required, disable grayscale mode. The platform automatically upgrades the remaining hosts and related resources.

    Regardless of the method you choose, continuously monitor VM instance status during the upgrade to ensure that services remain unaffected.

  14. Disable Grayscale Upgrade Mode

    After you confirm that the grayscale upgrade validation completes without issues, run the following command to disable the grayscale upgrade mode:

    [root@localhost ~]# zstack-cli   // Enter zstack-cli.
    admin >>> LogInByAccount accountName=admin password=<ADMIN_PASSWORD>     // Log in to the CLI with the admin account. <ADMIN_PASSWORD> is the password of the admin account and is used as a placeholder here.
    admin >>> UpdateGlobalConfig name=grayscaleUpgrade category=upgradeControl value=false     // Disable the grayscale mode.
  15. Enable VM HA globally.

    Login to the UI, choose Settings > Platform Setting > HA Policy, on the HA Policy page, click the Enable button on the upper right corner.

What to do next

Now, the platform grayscale upgrade is complete in a dual-management node deployment. You can continue to upgrade virtualization tools and Kernel as needed.

Upgrade Virtualization Tools

About this task

Different ZStack Cloud version may use different virtualization tools. After the cloud platform upgrading, the virtualization tools ( QEMU & Libvirt) are not updated automatically. To ensure the proper use of the latest features and security patches, you can upgrade virtualization tools manually after you upgrade the cloud platform.

Procedure

  1. Update the local repo of the management node(s).
    Make sure that you have update the local repo of the management node(s) with the new ISO. If not, run the following command on the management node(s). If you deploy dual management nodes, run the command on the two management nodes respectively:
    [root@localhost ~]# zstack-upgrade -r ZStack-Cloud-x86_64-DVD-5.5.24-h84r.iso
  2. Back up the database of the management node(s).
    Make sure that you have backed up the database of the management node(s). If not, run the following command on the management node(s) to backup the database to /var/lib/zstack/mysql-backup/lib/zstack/mysql-backup/. If you deploy dual managemnt nodes, run the command on the two management nodes respectively:
    [root@localhost ~]# zstack-ctl dump_mysql --file-name zstack-db-backup
  3. Check ZStack Cloud services and hardware connectivity.
    Make sure that the ZStack Cloud service is normal and all hardware devices are connected well.
  4. Disable the VM high availability globally.
    Log in to the ZStack Cloud UI. On the main menu, choose Setting > Platform Setting > HA Policy. On the HA Policy page, click the Disable button on the upper right corner.
  5. Enable the global setting zstack.experimental.repo.
    Run the following commands on the management node to enable the global setting zstack.experimental.repo. If you deploy dual management nodes, run on the management node that corresponds to the VIP:
    [root@localhost ~]# zstack-cli
    # Lanuch the CLI command tool
    - >>>  LogInByAccount accountName=admin password=${Password}
    # Log in the admin account.
    admin >>> UpdateGlobalConfig category=cluster name=zstack.experimental.repo value=true
    # Enable the global setting zstack.experimental.repo.
    Note: If you deploy dual management nodes, you can run zsha2 status on either of the management nodes. The management node whose VIP value is yes is the management node you need.
  6. Upgrade host QEMU.
    Run the following CLI commands on the management node to upgrade host QEMU:
    admin >>> QueryCluster    //Query UUIDs of clusters.
    admin >>> UpdateClusterOS updatePackages=qemu-kvm excludePackages=python2-crypto uuid=${ClusterUuid}    //Upgrade QEMU of hosts in the specified cluster. Record the uuid (LongJobUuid) returned.
    admin >>> QueryLongJob uuid=${LongJobUuid}    //Query the upgrading progress. If the returned state value is Succeeded, the upgrading is completed.
    Note: If you deploy more than one cluster, you can upgrade the QEMU of host in different clusters sequentially.
  7. Upgrade host Libvirt.
    Run the following CLI commands on the management node to upgrade host Libvirt:
    admin >>> QueryCluster    //Query UUIDs of clusters.
    admin >>> UpdateClusterOS updatePackages=glib2 uuid=${ClusterUuid}
    admin >>> UpdateClusterOS updatePackages=libvirt excludePackages=python2-crypto uuid=${ClusterUuid}    //Upgrade Libvirt of hosts in the specified cluster. Record the uuid (LongJobUuid) returned.
    admin >>> QueryLongJob uuid=${LongJobUuid}    //Query the upgrading progress. If the returned state value is Succeeded, the upgrading is completed.
    Note: If you deploy more than one cluster, you can upgrade the Libvirt of host in different clusters sequentially.
  8. Check the QEMU and Libvirt version after the upgrading.
    Run the following command on the hosts to check whether the QEMU and Libvirt are upgraded to the target version:
    [root@localhost ~]# qemu-img -V    //Check the QEMU version.
    [root@localhost ~]# rpm -qa | grep libvirt    //Check the Libvirt version.
  9. Disable the global setting zstack.experimental.repo.
    After the upgrading, run the following CLI command on the management node to disable the global setting zstack.experimental.repo:
    admin >>> UpdateGlobalConfig category=cluster name=zstack.experimental.repo value=false
  10. Reboot/migrate VM instances, VPC vRouters, and LB instances.
    Log in to the ZStack Cloud UI, reboot or migrate VM instances, VPC vRouters, and LB instances on the hosts whose QEMU is upgraded, making the QEMU version of VM instances, VPC vRouters, and LB instances consistent with the QEMU version of the hosts they are residing on.
  11. Enable the VM high availability globally.
    On the main menu of ZStack Cloud, choose Setting > Platform Setting > HA Policy. On the HA Policy page, click the Enable button on the upper right corner.

What to do next

Now, you have completed the virtualization tool (QEMU & Libvirt) upgrading.

Upgrade Kernel

About this task

Different ZStack Cloud version may use different Kernel. After the cloud platform upgrading, the host Kernel are not updated automatically. To ensure the proper use of the latest features and security patches, you can upgrade Kernel manually after you upgrade the cloud platform.

Procedure

  1. Update the local repo of the management node(s).
    Make sure that you have update the local repo of the management node(s) with the new ISO. If not, run the following command on the management node(s). If you deploy dual management nodes, run the command on the two management nodes respectively:
    [root@localhost ~]# zstack-upgrade -r ZStack-Cloud-x86_64-DVD-5.5.24-h84r.iso
  2. Back up the database of the management node(s).
    Make sure that you have backed up the database of the management node(s). If not, run the following command on the management node(s) to backup the database to /var/lib/zstack/mysql-backup/lib/zstack/mysql-backup/. If you deploy dual management nodes, run the command on the two management nodes respectively:
    [root@localhost ~]# zstack-ctl dump_mysql --file-name zstack-db-backup
  3. Check ZStack Cloud services and hardware connectivity.
    Make sure that the ZStack Cloud service is normal and all hardware devices are connected well.
  4. Disable the VM high availability globally.
    Log in to the ZStack Cloud UI. On the main menu, choose Setting > Platform Setting > HA Policy. On the HA Policy page, click the Disable button on the upper right corner.
  5. Enable the global setting zstack.experimental.repo.
    Run the following commands on the management node to enable the global setting zstack.experimental.repo. If you deploy dual management nodes, run on the management node that corresponds to the VIP:
    [root@localhost ~]# zstack-cli
    # Lanuch the CLI command tool
    - >>>  LogInByAccount accountName=admin password=${Password}
    # Log in the admin account.
    admin >>> UpdateGlobalConfig category=cluster name=zstack.experimental.repo value=true
    # Enable the global setting zstack.experimental.repo.
    Note: If you deploy dual management nodes, you can run zsha2 status on either of the management nodes. The management node whose VIP value is yes is the management node you need.
  6. Upgrade host Kernel.
    Run the following CLI commands on the management node to upgrade host Kernel:
    admin >>> QueryCluster    //Query UUIDs of clusters.
    admin >>> UpdateClusterOS updatePackages=kernel excludePackages=python2-crypto uuid=${ClusterUuid}    //Upgrade Kernel of hosts in the specified cluster. Record the uuid (LongJobUuid) returned.
    admin >>> QueryLongJob uuid=${LongJobUuid}    //Query the upgrading progress. If the returned state value is Succeeded, the upgrading is completed.
    Note: If you deploy more than one cluster, you can upgrade the Kernel of host in different clusters sequentially.
  7. Check the Kernel version after the upgrading.
    Run the following command on the hosts to check whether the Kernel are upgraded to the target version:
    [root@localhost ~]# rpm -qa | grep -i kernel
  8. Reboot hosts.
    Reboot hosts to make the new Kernel version take effect.
    Note: Rebooting a host temperately interrupts VM business running on it. We recommend that you perform this action during off-peak periods.
  9. Disable the global setting zstack.experimental.repo.
    After the upgrading, run the following CLI command on the management node to disable the global setting zstack.experimental.repo:
    admin >>> UpdateGlobalConfig category=cluster name=zstack.experimental.repo value=false
  10. Enable the VM high availability globally.
    On the main menu of ZStack Cloud, choose Setting > Platform Setting > HA Policy. On the HA Policy page, click the Enable button on the upper right corner.

What to do next

Now, you have completed the Kernel upgrading and the whole typical upgrading process of ZStack Cloud. If you have any other question or need upgrading help, contact the official technical support.