DocsKnowledge BaseAPI Explorer
Documentation›ZCF›ZStack Cloud›Explore more›Upgrade TutorialZCF

Overview

To upgrade ZStack Cloud, you need to upgrade the management nodes, compute nodes, storage nodes, and network nodes to the latest version and update basic software packages, software library dependencies, and core virtualization tools. This document describes how to upgrade ZStack Cloud in details, including:

Limitations

This document applies to cloud platform upgrading of the same repository (repo), which means:
  • The management nodes and all hosts on the cloud platform are using the same operating system (OS) version, for example, all use the h84r OS.
  • The OS version used by the target ISO is consistent with OS version currently used by the management nodes and hosts. For example, if all management nodes and hosts on the cloud platform are using the h84r OS, you must use a ZStack Cloud ISO of the h84r edition to perform the upgrading.

Upgrading Mechanism

ZStack Cloud provides the following upgrading mechanisms:
  • Seamless upgrading: In a same-repo environment, ZStack Cloud of any old version can be upgraded to the latest version.
  • Lossless upgrading: The upgrade process does not affect the services running in VM instances.
Note:
  • This document takes upgrading the cloud platform of same-repo with the h84r ISO as an example.
  • If you need to change host OS or upgrade a cloud platform of multiple repos, contact the official technical support for help.

Considerations

Before upgrading ZStack Cloud, be aware of the following points:
  • During the upgrading, the management nodes are inaccessible and you cannot apply VM and volume resources. Therefore, we recommend that you upgrade the cloud platform during business troughs to avoid resource application and business change needs.
  • If you upgrade the virtualization tools (QEMU & Libvirt), it is not supported to rollback the tools to earlier versions.
  • If you upgrade the host QEMU, you need to reboot or migrate VM instances, VPC vRouters, and LB instances on the hosts to make the new QEMU version take effect.
  • If you upgrade the host Kernel, you need to restart the host to make the new Kernel version take effect.
  • If any errors occur during the upgrading, interrupting the upgrading process or affecting the running business, contact the official technical support immediately.
  • Read the feature update information to learn about the key feature changes in the target version in advance.
Version Upgrade Information Details
ZStack Cloud 5.1.0 Removes the UI button Set VM RDP Mode. Starting from 5.1.0, it is not supported to set the VM RDP mode on the UI. To enable the RDP mode for a VM instance, run the CLI command SetVmRDP uuid=${vmuuid} enable=true
ZStack Cloud 4.7.21 Renames Entertainment Management as Tenant Management. Starting from 4.7.21, the feature module Entertainment Management is renamed as Tenant Management. This does not affect the functions of this module.
ZStack Cloud 4.6.0 Upgrades Affinity Group to VM Scheduling Policy. Starting from 4.6.0, the feature Affinity Group is upgraded to VM Scheduling Policy:
  • The VM Scheduling Policy feature provides four types of policies: VM Exclusive from Each Other, VM Affinitive to Each Other, VMs Affinitive to Hosts, and VMs Exclusive from Hosts. Each policy has two execution mechanisms: Hard and Soft.
  • Anti-Affinity Group (Hard) and Anti-Affinity Group (Soft) policies provided in earlier versions are upgraded to VM Affinitive to Each Other (Hard) and VM Affinitive to Each Other (Soft) respectively, without any modification to the underlying working principles. This upgrade has no impact on your existing application workloads and is annoyance-free.
ZStack Cloud 4.0.0 Upgrades vRouter to VPC vRouter. Starting from 4.0.0, vRouter is upgraded to VPC vRouter and vRouter network is upgraded to VPC network. vRouter is no longer available on the UI. The seamless upgrade does not affect your business continuity.
Replaces User Group and User by the account system in Entertainment Management. Starting from 4.0.0, User Group and User are replaced by the system account in Entertainment Management Account System. User Group and User are no longer available on the UI. You cannot login to the Cloud by using a user or user group account. Before you implement upgrading, migrate the user and user group account data to Entertainment Management.
Note: User accounts created by the admin and have the admin permissions are also can celled. Platform admins in Entertainment Management have the equivalent permissions. You can use Entertainment Management to create platform managers based on your business needs.
Adjusts the integration of AD/LDAP and accounts. Starting from 4.0.0, the integration of AD/LDAP and accounts is managed by Entertainment Management. Before you implement upgrading, migrate the AD/LDAP account data managed by the original account system to Entertainment Management.

Upgrade Cloud Platform

Manually upgrade the management nodes of ZStack Cloud, and all other nodes, such as hosts, primary storage, image storage, and VPC vRouters, will be automatically reconnected and upgraded.

ZStack Cloud provides single-management node and dual-management node deployment methods. The two deployment methods have different upgrading processes.

Single-Management Node Upgrading

About this task

This topic describes the upgrading process of a single-management node platform.

Procedure

  1. Obtain software packages for upgrading.
    Prepare the following packages and save them in the /root/ directory of the management node:
    • ZStack Cloud ISO:
      • Software: ZStack-Cloud-x86_64-DVD-5.4.12-h84r.iso
      • Download address: Click here
    • ZStack Cloud installation package:
      • Software: ZStack-Cloud-installer-5.4.12.bin
      • Download address: Click here
    • ZStack Cloud Upgrade Script:
      • Software: ZStack-Cloud-upgrade
      • Download address: Click here
    Note:
    • ZStack-Cloud supports offline upgrading, which allows you to upgrade the cloud platform without connecting to the public network. Therefore, you need to download the software packages and upgrade script in advance.
    • After you download the software packages and upgrade script, check the MD5 hash of the packages or scripts. If the MD5 hash is inconsistent with that on the official website, download again.
  2. Disable VM HA globally.

    Login to the UI, choose Settings > Platform Setting > HA Policy, on the HA Policy page, click the Disable button on the upper right corner.

  3. Back up the database and upgrading script.
    Before you upgrade the cloud platform, back up the database and upgrading script, in case of any unexpected situations.
    • Run the following command on the management node to back up the database in the /var/lib/zstack/mysql-backup/ directory:
      [root@localhost ~]# zstack-ctl dump_mysql --file-name zstack-db-backup
    • Run the following command on the management node to back up the upgrading script in the /root/ directory:
      [root@localhost ~]# cp /usr/local/bin/zstack-upgrade /root/zstack-upgrade-bk
  4. Upgrade the cloud platform.
    Run the following commands on the management node to implement offline upgrading:
    # Two offline upgrading methods are supported. The first one is recommended:
# First method: Upgrade the local repo and management service simultaneously:
[root@zstack-1 opt]# zstack-upgrade ZStack-Cloud-x86_64-DVD-5.4.12-h84r.iso
# Second method: Upgrade the local repo first and then the management service:
[root@localhost ~]# zstack-upgrade -r ZStack-Cloud-x86_64-DVD-5.4.12-h84r.iso
[root@localhost ~]# bash ZStack-Cloud-installer-5.4.12.bin -u -P MYSQL_ROOT_PASSWORD
    Note:
    • If the root password of the database is the default password, you do not need to specify -P MYSQL_ROOT_PASSWORD.
    • Before you run the zstack-upgrade command, make sure that you use ISO of the h84r version instead of other versions.
    • While upgrading the local repo, the upgrade script is auto-updated. When the update is completed, rerun the preceding command.
  5. Clean up the browser cache.

    To make sure that the upgraded platform works as expected, after the upgrading is completed, you need to login to the UI platform and press Ctrl+F5 or Ctrl+Shift+R to clean up the browser cache.

  6. Enable VM HA globally.

    Login to the UI, choose Settings > Platform Setting > HA Policy, on the HA Policy page, click the Enable button on the upper right corner.

What to do next

  • Now, the management node has been upgraded. The system will automatically reconnects other nodes, such as hosts, primary storage, image storage, and VPC vRouters, to complete the cloud platform upgrading.
  • You can continue to upgrade virtualization tools and Kernel as needed.

Dual-Management Node Upgrading

About this task

This topic describes the upgrading process of a dual-management node platform.

Procedure

  1. Obtain software packages for upgrading.
    Prepare the following packages and save them respectively in the /root/ directory of the two management nodes:
    • ZStack Cloud ISO:
      • Software: ZStack-Cloud-x86_64-DVD-5.4.12-h84r.iso
      • Download address: Click here
    • ZStack Cloud installation package:
      • Software: ZStack-Cloud-installer-5.4.12.bin
      • Download address: Click here
    • ZStack Cloud Upgrade Script:
      • Software: ZStack-Cloud-upgrade
      • Download address: Click here
    • Multi-MN-HA Suite:
      • Software: ZStack-Multinode-HA-Suite-5.4.12.tar.gz
      • Download address: Click here
    Note:
    • ZStack-Cloud supports offline upgrading, which allows you to upgrade the cloud platform without connecting to the public network. Therefore, you need to download the software packages and upgrade script in advance.
    • After you download the software packages and upgrade script, check the MD5 hash of the packages or scripts. If the MD5 hash is inconsistent with that on the official website, download again.
  2. Disable VM HA globally.

    Login to the UI, choose Settings > Platform Setting > HA Policy, on the HA Policy page, click the Disable button on the upper right corner.

  3. Back up the database and upgrading script.
    Before you upgrade the cloud platform, back up the database and upgrading script, in case of any unexpected situations.
    • Run the following command on the management nodes respectively to back up the database in the /var/lib/zstack/mysql-backup/ directory:
      [root@localhost ~]# zstack-ctl dump_mysql --file-name zstack-db-backup
    • Run the following command on the management nodes respectively to back up the upgrading script in the /root/ directory:
      [root@localhost ~]# cp /usr/local/bin/zstack-upgrade /root/zstack-upgrade-bk
  4. Update the local repo.
    Run the following command on the management nodes respectively to update the local repo:
    [root@localhost ~]# bash zstack-upgrade -r ZStack-Cloud-x86_64-DVD-5.4.12-h84r.iso
    Note: While upgrading the local repo, the upgrade script is auto-updated. When the update is completed, rerun the preceding command.
  5. Obtain the IP address of the management node that corresponds to the VIP you use.
    Run the following command to obtain the IP address of the management node that corresponds to the VIP you use. The management node that corresponds to the yes result is the management node you need:
    [root@localhost ~]# zsha2 status
  6. Decompress the Multi-MN-HA file and grant the decompressed files execution permission.
    Run the following commands on the management node that corresponds to the VIP you use to decompress the Multi-MN-HA file and grant the decompressed files zsha2 and zstack-hamon execution permission:
    [root@localhost ~]# tar zxvf ZStack-Multinode-HA-Suite-5.4.12.tar.gz      #Decompress the Multi-MN-HA file
[root@localhost ~]# chmod +x zsha2 zstack-hamon    #Grant the decompressed files execution permission
  7. Upgrade Multi-MN-HA Suite
    Run the following command on the management node that corresponds to the VIP you use to upgrade Multi-MN-HA Suite:
    [root@localhost ~]# ./zsha2 upgrade-ha
  8. Upgrade the cloud platform.
    Run the following command on the management node that corresponds to the VIP you use to upgrade the management nodes:
    zsha2 upgrade-mn -peerpass password ZStack-Cloud-installer-5.4.12.bin
    Note: -peerpass is optional. You can use the parameter to set the password of SSH login to the peer management nodes.
  9. Clean up the browser cache.

    To make sure that the upgraded platform works as expected, after the upgrading is completed, you need to login to the UI platform and press Ctrl+F5 or Ctrl+Shift+R to clean up the browser cache.

  10. Enable VM HA globally.

    Login to the UI, choose Settings > Platform Setting > HA Policy, on the HA Policy page, click the Enable button on the upper right corner.

What to do next

  • Now, the management nodes have been upgraded. The system will automatically reconnects other nodes, such as hosts, primary storage, image storage, and VPC vRouters, to complete the cloud platform upgrading.
  • You can continue to upgrade virtualization tools and Kernel as needed.

Upgrade Virtualization Tools

About this task

Different ZStack Cloud version may use different virtualization tools. After the cloud platform upgrading, the virtualization tools ( QEMU & Libvirt) are not updated automatically. To ensure the proper use of the latest features and security patches, you can upgrade virtualization tools manually after you upgrade the cloud platform.

Procedure

  1. Update the local repo of the management node(s).
    Make sure that you have update the local repo of the management node(s) with the new ISO. If not, run the following command on the management node(s). If you deploy dual management nodes, run the command on the two management nodes respectively:
    [root@localhost ~]# zstack-upgrade -r ZStack-Cloud-x86_64-DVD-5.4.12-h84r.iso
  2. Back up the database of the management node(s).
    Make sure that you have backed up the database of the management node(s). If not, run the following command on the management node(s) to backup the database to /var/lib/zstack/mysql-backup/lib/zstack/mysql-backup/. If you deploy dual managemnt nodes, run the command on the two management nodes respectively:
    [root@localhost ~]# zstack-ctl dump_mysql --file-name zstack-db-backup
  3. Check ZStack Cloud services and hardware connectivity.
    Make sure that the ZStack Cloud service is normal and all hardware devices are connected well.
  4. Disable the VM high availability globally.
    Log in to the ZStack Cloud UI. On the main menu, choose Setting > Platform Setting > HA Policy. On the HA Policy page, click the Disable button on the upper right corner.
  5. Enable the global setting zstack.experimental.repo.
    Run the following commands on the management node to enable the global setting zstack.experimental.repo. If you deploy dual management nodes, run on the management node that corresponds to the VIP:
    [root@localhost ~]# zstack-cli
# Lanuch the CLI command tool
- >>>  LogInByAccount accountName=admin password=${Password}
# Log in the admin account.
admin >>> UpdateGlobalConfig category=cluster name=zstack.experimental.repo value=true
# Enable the global setting zstack.experimental.repo.
    Note: If you deploy dual management nodes, you can run zsha2 status on either of the management nodes. The management node whose VIP value is yes is the management node you need.
  6. Upgrade host QEMU.
    Run the following CLI commands on the management node to upgrade host QEMU:
    admin >>> QueryCluster    //Query UUIDs of clusters.
admin >>> UpdateClusterOS updatePackages=qemu-kvm excludePackages=python2-crypto uuid=${ClusterUuid}    //Upgrade QEMU of hosts in the specified cluster. Record the uuid (LongJobUuid) returned.
admin >>> QueryLongJob uuid=${LongJobUuid}    //Query the upgrading progress. If the returned state value is Succeeded, the upgrading is completed.
    Note: If you deploy more than one cluster, you can upgrade the QEMU of host in different clusters sequentially.
  7. Upgrade host Libvirt.
    Run the following CLI commands on the management node to upgrade host Libvirt:
    admin >>> QueryCluster    //Query UUIDs of clusters.
admin >>> UpdateClusterOS updatePackages=glib2 uuid=${ClusterUuid}
admin >>> UpdateClusterOS updatePackages=libvirt excludePackages=python2-crypto uuid=${ClusterUuid}    //Upgrade Libvirt of hosts in the specified cluster. Record the uuid (LongJobUuid) returned.
admin >>> QueryLongJob uuid=${LongJobUuid}    //Query the upgrading progress. If the returned state value is Succeeded, the upgrading is completed.
    Note: If you deploy more than one cluster, you can upgrade the Libvirt of host in different clusters sequentially.
  8. Check the QEMU and Libvirt version after the upgrading.
    Run the following command on the hosts to check whether the QEMU and Libvirt are upgraded to the target version:
    [root@localhost ~]# qemu-img -V    //Check the QEMU version.
[root@localhost ~]# rpm -qa | grep libvirt    //Check the Libvirt version.
  9. Disable the global setting zstack.experimental.repo.
    After the upgrading, run the following CLI command on the management node to disable the global setting zstack.experimental.repo:
    admin >>> UpdateGlobalConfig category=cluster name=zstack.experimental.repo value=false
  10. Reboot/migrate VM instances, VPC vRouters, and LB instances.
    Log in to the ZStack Cloud UI, reboot or migrate VM instances, VPC vRouters, and LB instances on the hosts whose QEMU is upgraded, making the QEMU version of VM instances, VPC vRouters, and LB instances consistent with the QEMU version of the hosts they are residing on.
  11. Enable the VM high availability globally.
    On the main menu of ZStack Cloud, choose Setting > Platform Setting > HA Policy. On the HA Policy page, click the Enable button on the upper right corner.

What to do next

Now, you have completed the virtualization tool (QEMU & Libvirt) upgrading.

Upgrade Kernel

About this task

Different ZStack Cloud version may use different Kernel. After the cloud platform upgrading, the host Kernel are not updated automatically. To ensure the proper use of the latest features and security patches, you can upgrade Kernel manually after you upgrade the cloud platform.

Procedure

  1. Update the local repo of the management node(s).
    Make sure that you have update the local repo of the management node(s) with the new ISO. If not, run the following command on the management node(s). If you deploy dual management nodes, run the command on the two management nodes respectively:
    [root@localhost ~]# zstack-upgrade -r ZStack-Cloud-x86_64-DVD-5.4.12-h84r.iso
  2. Back up the database of the management node(s).
    Make sure that you have backed up the database of the management node(s). If not, run the following command on the management node(s) to backup the database to /var/lib/zstack/mysql-backup/lib/zstack/mysql-backup/. If you deploy dual management nodes, run the command on the two management nodes respectively:
    [root@localhost ~]# zstack-ctl dump_mysql --file-name zstack-db-backup
  3. Check ZStack Cloud services and hardware connectivity.
    Make sure that the ZStack Cloud service is normal and all hardware devices are connected well.
  4. Disable the VM high availability globally.
    Log in to the ZStack Cloud UI. On the main menu, choose Setting > Platform Setting > HA Policy. On the HA Policy page, click the Disable button on the upper right corner.
  5. Enable the global setting zstack.experimental.repo.
    Run the following commands on the management node to enable the global setting zstack.experimental.repo. If you deploy dual management nodes, run on the management node that corresponds to the VIP:
    [root@localhost ~]# zstack-cli
# Lanuch the CLI command tool
- >>>  LogInByAccount accountName=admin password=${Password}
# Log in the admin account.
admin >>> UpdateGlobalConfig category=cluster name=zstack.experimental.repo value=true
# Enable the global setting zstack.experimental.repo.
    Note: If you deploy dual management nodes, you can run zsha2 status on either of the management nodes. The management node whose VIP value is yes is the management node you need.
  6. Upgrade host Kernel.
    Run the following CLI commands on the management node to upgrade host Kernel:
    admin >>> QueryCluster    //Query UUIDs of clusters.
admin >>> UpdateClusterOS updatePackages=kernel excludePackages=python2-crypto uuid=${ClusterUuid}    //Upgrade Kernel of hosts in the specified cluster. Record the uuid (LongJobUuid) returned.
admin >>> QueryLongJob uuid=${LongJobUuid}    //Query the upgrading progress. If the returned state value is Succeeded, the upgrading is completed.
    Note: If you deploy more than one cluster, you can upgrade the Kernel of host in different clusters sequentially.
  7. Check the Kernel version after the upgrading.
    Run the following command on the hosts to check whether the Kernel are upgraded to the target version:
    [root@localhost ~]# rpm -qa | grep -i kernel
  8. Reboot hosts.
    Reboot hosts to make the new Kernel version take effect.
    Note: Rebooting a host temperately interrupts VM business running on it. We recommend that you perform this action during off-peak periods.
  9. Disable the global setting zstack.experimental.repo.
    After the upgrading, run the following CLI command on the management node to disable the global setting zstack.experimental.repo:
    admin >>> UpdateGlobalConfig category=cluster name=zstack.experimental.repo value=false
  10. Enable the VM high availability globally.
    On the main menu of ZStack Cloud, choose Setting > Platform Setting > HA Policy. On the HA Policy page, click the Enable button on the upper right corner.

What to do next

Now, you have completed the Kernel upgrading and the whole typical upgrading process of ZStack Cloud. If you have any other question or need upgrading help, contact the official technical support.
PreviousQuick Installation and Usage TutorialNextLicense Update Guide
Upgrade Tutorial | 5.4.12 | ZStack Cloud · ZCF | ZStack Resource Center