DocsKnowledge BaseAPI Explorer
Documentation›ZCF›ZStack Cloud›Explore more›Upgrade TutorialZCF

Overview

To upgrade ZStack Cloud, you need to upgrade the management node, compute nodes, storage nodes, and network nodes to the latest version and update basic software packages, software library dependencies, and core virtualization tools. This document describes how to upgrade ZStack Cloud in details.

Limitations

This document applies to cloud platform upgrading of the same repo, which means that the operating system (OS) version of the hosts is consistent with the management nodes. For example, both the hosts and management nodes use h84r OS.
Note: If you need to upgrade host OS or upgrade the cloud platform of multiple repos, contact the official technical support for help.

Upgrading Mechanism

ZStack Cloud provides the following upgrading mechanisms:
  • Seamless upgrading: seamlessly upgrades ZStack Cloud of any old version to the latest.
  • Lossless upgrading: does not affect the services that VM instances are running during ZStack Cloud upgrading.

Considerations

When you upgrade ZStack Cloud, note that:
  • During the upgrading, the management nodes are inaccessible and you cannot apply VM and volume resources. Therefore, we recommend that you upgrade the cloud platform during business troughs to avoid resource application and business change needs.
  • After the management node upgrading is completed, all hosts, primary storages, backup storages, VPC vRouters are reconnected and related resources are auto-upgraded. During the connection, you cannot perform operations on VM instances and volumes.
  • If an error occurs during the upgrading and interrupts the upgrading, you can troubleshoot the error. After the error is solved, you can upgrade the cloud platform again. If you have any question or need a help, contact the official technical support.
Version Upgrade Information Details
ZStack Cloud 4.8.10 Removes the UI button Set VM RDP Mode. Starting from 4.8.10, it is no longer supported to set the VM RDP mode on the UI. To enable RDP for a VM instance, you can run the CLI command SetVmRDP uuid=${vmuuid} enable=true
ZStack Cloud 4.7.21 Renames Entertainment Management as Tenant Management. Starting from 4.7.21, the feature module Entertainment Management is renamed as Tenant Management. This does not affect the functions of this module.
ZStack Cloud 4.6.0 Upgrades Affinity Group to VM Scheduling Policy. Starting from 4.6.0, the feature Affinity Group is upgraded to VM Scheduling Policy:
  • The VM Scheduling Policy feature provides four types of policies: VM Exclusive from Each Other, VM Affinitive to Each Other, VMs Affinitive to Hosts, and VMs Exclusive from Hosts. Each policy has two execution mechanisms: Hard and Soft.
  • Anti-Affinity Group (Hard) and Anti-Affinity Group (Soft) policies provided in earlier versions are upgraded to VM Affinitive to Each Other (Hard) and VM Affinitive to Each Other (Soft) respectively, without any modification to the underlying working principles. This upgrade has no impact on your existing application workloads and is annoyance-free.
ZStack Cloud 4.0.0 Upgrades vRouter to VPC vRouter. Starting from 4.0.0, vRouter is upgraded to VPC vRouter and vRouter network is upgraded to VPC network. vRouter is no longer available on the UI. The seamless upgrade does not affect your business continuity.
Replaces User Group and User by the account system in Entertainment Management. Starting from 4.0.0, User Group and User are replaced by the system account in Entertainment Management Account System. User Group and User are no longer available on the UI. You cannot login to the Cloud by using a user or user group account. Before you implement upgrading, migrate the user and user group account data to Entertainment Management.
Note: User accounts created by the admin and have the admin permissions are also can celled. Platform admins in Entertainment Management have the equivalent permissions. You can use Entertainment Management to create platform managers based on your business needs.
Adjusts the integration of AD/LDAP and accounts. Starting from 4.0.0, the integration of AD/LDAP and accounts is managed by Entertainment Management. Before you implement upgrading, migrate the AD/LDAP account data managed by the original account system to Entertainment Management.

Preparations

When you upgrade ZStack Cloud, you may need the following software packages. Before the upgrading, prepare the packages based on your needs:
  • ZStack Cloud ISO:
    • h84r: ZStack-Cloud-x86_64-DVD-4.8.38-h84r.iso
    • Download address: Click here
  • ZStack Cloud installation package:
    • Software: ZStack-Cloud-installer-4.8.38.bin
    • Download address: Click here
  • Multi-MN-HA:
    • Software: ZStack-Multinode-HA-Suite-4.8.38.tar.gz
    • Download address: Click here
Note: After you download the software packages or scripts, check the MD5 hash of the packages or scripts. If the MD5 hash is inconsistent with that on the official website, download again.

Typical Upgrading Processes

This topic describes the detailed upgrading processes of ZStack Cloud, including the cloud platform upgrading and core virtualization tool upgrading (if needed).
  • Cloud platform upgrading: Manually upgrade the management nodes. After the management node upgrading, all hosts, primary storage, image storage, VPC vRouters are reconnected and related resources are auto-upgraded.
  • Virtualization tool upgrading: Upgrading an existing cloud platform with a new ISO does not auto-update the virtualization tools. However, to ensure the proper use of the latest features and security patches, we recommend that you upgrade virtualization tools to the version corresponding to the new ISO manually after the cloud platform upgrading.

Upgrade Cloud Platform

ZStack Cloud provides single-management node and multi-management node deployment methods. The two deployment methods have different upgrading processes.

Single-Management Node Upgrading

About this task

This topic takes h84r ISO as an example to describe the upgrading process of a single-management node platform.

Procedure

  1. Obtain software packages for upgrading.
    See Preparations to obtain the packages you need and save the packages in the /root/ directory.
    • Offline upgrading does not require connection to the public network but requires the bin package, ISO package, and upgrade script.
  2. Disable VM HA globally.

    Log in to the UI, choose Settings > Platform Setting > HA Policy, on the HA Policy page, click the Disable button on the upper right corner.

  3. Back up the database and upgrading script.
    Before you upgrade the cloud platform, back up the database and upgrading script, in case of any unexpected situations.
    • Run the following command on the management node to back up the database in the /var/lib/zstack/mysql-backup/ directory:
      [root@localhost ~]# zstack-ctl dump_mysql --file-name zstack-db-backup
    • Run the following command on the management node to back up the upgrading script in the /root/ directory:
      [root@localhost ~]# cp /usr/local/bin/zstack-upgrade /root/zstack-upgrade-bk
  4. Upgrade the cloud platform.
    Run the following commands on the management node to implement offline upgrading:
    # Two offline upgrading methods are supported. The first one is recommended:
# First method: Upgrade the local repo and management service simultaneously:
[root@zstack-1 opt]# zstack-upgrade ZStack-Cloud-x86_64-DVD-4.8.38-h84r.iso
# Second method: Upgrade the local repo first and then the management service:
[root@localhost ~]# zstack-upgrade -r ZStack-Cloud-x86_64-DVD-4.8.38-h84r.iso
[root@localhost ~]# bash ZStack-Cloud-installer-4.8.38.bin -u -P MYSQL_ROOT_PASSWORD
    Note:
    • If the root password of the database is the default password, you do not need to specify -P MYSQL_ROOT_PASSWORD.
    • Before you run the zstack-upgrade command, make sure that you use ISO of the h84r version instead of other versions.
    • While upgrading the local repo, the upgrade script is auto-updated. When the update is completed, rerun the preceding command.
  5. Clean up the browser cache.

    To make sure that the upgraded platform works as expected, after the upgrading is completed, you need to log in to the UI platform and press Ctrl+F5 or Ctrl+Shift+R to clean up the browser cache.

  6. Enable VM HA globally.

    Login to the UI, choose Settings > Platform Setting > HA Policy, on the HA Policy page, click the Enable button on the upper right corner.

Multi-Management Node Upgrading

About this task

This topic takes h84r ISO as an example to describe the upgrading process of a multi-management-node platform.

Procedure

  1. Obtain software packages for upgrading.

    See Preparations to obtain the ISO package, bin package, upgrading script, multi-MN-HA you need and save the packages or scripts in the /root/ directory of the management nodes.

  2. Disable VM HA globally.

    Log in to the UI, choose Settings > Platform Setting > HA Policy, on the HA Policy page, click the Disable button on the upper right corner.

  3. Back up the database and upgrading script.
    Before you upgrade the cloud platform, back up the database and upgrading script, in case of any unexpected situations.
    • Run the following command on the management nodes respectively to back up the database in the /var/lib/zstack/mysql-backup/ directory:
      [root@localhost ~]# zstack-ctl dump_mysql --file-name zstack-db-backup
    • Run the following command on the management nodes respectively to back up the upgrading script in the /root/ directory:
      [root@localhost ~]# cp /usr/local/bin/zstack-upgrade /root/zstack-upgrade-bk
  4. Update the local repo.
    Run the following command on the management nodes respectively to update the local repo:
    [root@localhost ~]# bash zstack-upgrade -r ZStack-Cloud-x86_64-DVD-4.8.38-h84r.iso
    Note: While upgrading the local repo, the upgrade script is auto-updated. When the update is completed, rerun the preceding command.
  5. Obtain the IP address of the management node that corresponds to the VIP you use.
    Run the following command to obtain the IP address of the management node that corresponds to the VIP you use. The management node that corresponds to the yes result is the management node you need:
    [root@localhost ~]# zsha2 status
  6. Decompress the Multi-MN-HA file and grant the decompressed files execution permission.
    Run the following commands on the management node that corresponds to the VIP you use to decompress the Multi-MN-HA file and grant the decompressed files zsha2 and zstack-hamon execution permission:
    [root@localhost ~]# tar zxvf ZStack-Multinode-HA-Suite-4.8.38.tar.gz      #Decompress the Multi-MN-HA file
[root@localhost ~]# chmod +x zsha2 zstack-hamon    #Grant the decompressed files execution permission
  7. Upgrade Multi-MN-HA
    Run the following command on the management node that corresponds to the VIP you use to upgrade Multi-MN-HA:
    [root@localhost ~]# ./zsha2 upgrade-ha
  8. Upgrade the cloud platform.
    Run the following command on the management node that corresponds to the VIP you use to upgrade the management nodes:
    zsha2 upgrade-mn -peerpass password ZStack-Cloud-installer-4.8.38.bin
    Note: -peerpass is optional. You can use the parameter to set the password of SSH login to the peer management nodes.
  9. Clean up the browser cache.

    To make sure that the upgraded platform works as expected, after the upgrading is completed, you need to log in to the UI platform and press Ctrl+F5 or Ctrl+Shift+R to clean up the browser cache.

  10. Enable VM HA globally.

    Login to the UI, choose Settings > Platform Setting > HA Policy, on the HA Policy page, click the Enable button on the upper right corner.

Upgrade Virtualization Tools

Host QEMU Upgrading

About this task

After upgrading the cloud platform with the latest ISO, you can batch upgrade the QEMU of hosts within the same cluster to the version that matches the new ISO.
Follow these steps:
  1. Make sure that the local repo of the management node(s) is upgraded based on the new ISO.
  2. Make sure that the database of the management node(s) is backed up.
  3. Check the ZStack Cloud service and hardware connectivity.
  4. Disable the VM high availability globally.
  5. Enable the global setting zstack.experimental.repo.
  6. Upgrade host QEMU.
  7. Check the upgrading result.
  8. Disable the global setting zstack.experimental.repo.
  9. Enable the VM high availability globally.

Procedure

  1. Make sure that the local repo of the management node(s) is upgraded based on the new ISO.
    Make sure that you have run the following command on the management node(s) to upgrade the local repo with the new ISO:
    [root@localhost ~]# zstack-upgrade -r ZStack-Cloud-x86_64-DVD-4.8.38-h84r.iso
  2. Make sure that the database of the management node(s) is backed up.
    Make sure that you have backed up the database of the management node(s) before the upgrading to /var/lib/zstack/mysql-backup/lib/zstack/mysql-backup/ for the recovery if an error occur.
    If not, run the following commands on the management node(s) to complete the backup:
    [root@localhost ~]# zstack-ctl dump_mysql --file-name zstack-db-backup
  3. Check ZStack Cloud services and hardware connectivity.
    Make sure that the ZStack Cloud service is normal and all hardware devices are connected well.
  4. Disable the VM high availability globally.
    On the main menu of ZStack Cloud, choose Setting > Platform Setting > HA Policy. On the HA Policy page, click the Disable button on the upper right corner.
  5. Enable the global setting zstack.experimental.repo.
    Run the following commands on the management node(s) to enable the global setting zstack.experimental.repo. If your Cloud has two managements node, run on the management node whos IP address is corresponding to the Cloud VIP:
    [root@localhost ~]# zstack-cli
# Lanuch the CLI command tool.- >>>  LogInByAccount accountName=admin password=${Password}
# Log in the admin account.
admin >>> UpdateGlobalConfig category=cluster name=zstack.experimental.repo value=true
# Enable the global setting zstack.experimental.repo.
  6. Upgrade host QEMU.
    Run the following CLI commands on the management node to batch upgrade QEMU of hosts within the same cluster:
    admin >>> UpdateClusterOS updatePackages=qemu-kvm-ev uuid=${ClusterUUID}
    Record the LongJob UUID returned.
    Figure 1. LongJob UUID


  7. Check the upgrading result.
    1. Check whether the upgrading is finished.
      Run the following CLI command on the management node, using the LongJob UUID returned in the previous step to check whether the upgrading is finished:
      admin >>> QueryLongJob uuid=${LongJobUUID}
      The returned state value ss Succeeded means the upgrading is finished.
    2. Check whether the host QEMU is upgraded to the version that matches the new ISO.
      Run the following CLI command on the management node to check whether the host QEMU version match the new ISO version: 
      admin >>> GetVirtualizerInfo uuids=${HostUUID}
      The returned matchState value as Matched means that the host QEMU version match the new ISO version.
      Figure 2. Check Host QEMU Version after Upgrading


  8. Disable the global setting zstack.experimental.repo.
    After the upgrading, run the following CLI command on the management node to disable the global setting zstack.experimental.repo:
    admin >>> UpdateGlobalConfig category=cluster name=zstack.experimental.repo value=false
  9. Enable the VM high availability globally.
    On the main menu of ZStack Cloud, choose Setting > Platform Setting > HA Policy. On the HA Policy page, click the Enable button on the upper right corner.

What to do next

Now, you have upgraded host QEMU to the version matching the new ISO. To make the new QEMU take effect, you need to manually reboot or migrate the VM instances, VPC vRouters, and LB instances on the hosts to synchronously upgrade the QEMU of these cloud resources.
PreviousQuick Installation and Usage Tutorial