Preparations

Creating a Cluster

In the ZStack Zaku main menu, click Cluster Management > Cluster Management > Cluster to enter the Cluster page. Click Create Cluster to enter the Create Cluster page.

Creating a cluster includes the following three steps:

1. Node configuration
   You can refer to the following example to enter the required information:

   • Reuse the Business Network as the Management Network: select whether the business network is reused as the management network. After it is enabled, the management network reuses the business network, and the management cluster accesses the nodes and manages the business cluster through the business network IP addresses of the nodes. Make sure that the management cluster and the business network are connected.
   • Cluster HA: select whether to enable cluster HA. It is enabled by default. If it is disabled, the cluster does not support HA and only 1 control plane node is required. It is not recommended to disable cluster HA in a production environment.
   • Node Information: enter the information of the nodes to be used to create the cluster.
     • Adding Method: the method of adding nodes, including IP address and IP range.
     • Business Network IPv4 Address: enter the business network IPv4 address of the node. Enter this field when the adding method is IP address.
     • Management Network IPv4 Address: enter the management network IPv4 address of the node. Enter this field when the adding method is IP address. The management cluster accesses and manages the business cluster nodes through the management network IPv4 addresses, so make sure that the management cluster and this address are connected. When the business network is reused as the management network, the management network IPv4 address is the same as the business network IPv4 address.
     • Business Network IPv4 Range: enter the business network IPv4 range of the nodes. Enter this field when the adding method is IP range.
     • Management Network IPv4 Range: enter the management network IPv4 range of the nodes. Enter this field when the adding method is IP range. When the business network is reused as the management network, the management network IPv4 range is the same as the business network IPv4 range.
       Note:

       • Make sure that the IP addresses entered are consistent with the actual configuration of the machines.
       • Make sure that the management network IPv4 addresses and the management cluster are connected. When the business network is reused as the management network, make sure that the business network IPv4 addresses and the management cluster are connected.
       • Make sure that the business network IPv4 addresses of the nodes are connected to each other.
       • The management network IPv4 addresses of different nodes cannot be duplicated. The business network IPv4 addresses of different nodes cannot be duplicated. When the business network is not reused as the management network, a management network IPv4 address and a business network IPv4 address cannot be duplicated.
       • When the business network is not reused as the management network, make sure that the IP addresses in the management network IPv4 range correspond to those in the business network IPv4 range one by one. The IP addresses in the same order belong to the same node. For example, the 1st IP address in the business network IPv4 range and the 1st IP address in the management network IPv4 range belong to the 1st node.
       • Make sure that the names of the NICs where the business network IPv4 addresses of the 3 control plane nodes are located are the same, and that the names of the NICs where the management network IPv4 addresses of the 3 control plane nodes are located are the same.
     • Node Name: set the name of the node. Naming rules: the length is limited to 2 to 45 characters, and the input can contain only lowercase letters, digits and the separator (-). It must start with a lowercase letter and cannot end with the separator. When the adding method is IP range, a start suffix must be specified. The node name consists of the name and the suffix, and the suffix increases from the start suffix in turn, such as "-1/-2/-3".
       Note:

       • The hostname of a host is set to the node name during the process of adding the node.
       • The node names used in this batch cannot be duplicated.
     • Node Role: select the role of the node. Control plane node, compute node and GPU node are available. When cluster HA is enabled, a cluster must have 3 control plane nodes. When cluster HA is disabled, a cluster must have 1 control plane node.
       Note:

       • A control plane node can also be used as a compute node, and then the business pods can also be scheduled to the control plane node.
       • The corresponding components are installed automatically on a GPU node during the deployment process. Select whether to install a machine as a GPU node according to the actual situation.
       • If GPU node is selected, make sure that the GPU driver has been installed correctly on the node. If the driver is not installed, only the corresponding components are installed and the node is not set to the GPU node role.
     • GPU Manufacturer: this field must be entered when GPU node is selected as the node role. Select the manufacturer of the GPUs on the node, and the system installs the GPU-related components of the corresponding manufacturer. Currently the following manufacturers are supported: NVIDIA, Huawei, HYGON, Iluvatar and Alibaba
       Note:

       • The huawei NPU driver and related components require the system accounts HwHiAiUser and hwMindX, and UID/GID 1000 and 9000. A UID/GID conflict message means that UID/GID 1000 or 9000 on the listed nodes is already used by users or groups other than these huawei accounts.
       • If huawei is selected as the GPU manufacturer, resolve the conflicts on the listed nodes and run the check again. You can continue only after the check passes.
       • If huawei is not selected as the GPU manufacturer, you can acknowledge the risk and continue. However, resolve the conflicts before using huawei NPUs later, or the driver or related components may fail to install.
   • Iluvatar GPU Virtualization License: this field must be entered when there is a GPU node and the GPU manufacturer is Iluvatar. Enter the license of Iluvatar GPU virtualization. This license is required to share Iluvatar GPUs in containers, and it is obtained from Iluvatar officially.
   • Node ROOT Password: enter the password of the root account of the nodes, which is used to connect to the nodes over SSH to execute the installation steps during the process of adding the nodes. Make sure that the ROOT password of each node is the same.
   • Node SSH Port: enter the port used to connect to the nodes over SSH, which is used to connect to the nodes over SSH to execute the installation steps during the process of adding the nodes. Make sure that this port of each node can be connected.
2. Basic configuration
   You can refer to the following example to enter the required information:

   • Cluster Name: set the name of the cluster. Naming rules: the length is limited to 2 to 50 characters, and the input can contain only lowercase letters, digits and the separator (-). It must start with a lowercase letter and cannot end with the separator.
   • Cluster Version: select the Kubernetes version of the cluster. The default is version 1.35, and version 1.30 can be enabled by configuration.
   • Business Network VIP: enter the high-availability address of the cluster API Server on the business network. Use an IPv4 address within the business network segment.
     • Cloud Network for the Business Network: if the platform is connected to ZStack Cloud 5.2.0 or later and a cloud network contains this VIP, select that network. The platform reserves the IP address in the selected network to prevent other cloud platform resources from using it.
   • Management Network VIP: enter the high-availability address of the cluster API Server on the management network. Use an IPv4 address within the management network segment. If the business network is reused as the management network, both networks use the same VIP.
     • Cloud Network for the Management Network: if the business network is not reused as the management network, select the network containing the management network VIP under the conditions described above. The platform reserves the IP address in the selected network.
   • Maximum Number of Pods per Node: enter the maximum number of pods allowed to run on each node. The default is 110, and only an integer from 50 to 1000 can be entered.
   • Monitoring and Log Disk: the monitoring and log disk is used by the monitoring and log components to create PVCs and store the monitoring and log data. Select 1 disk for each control plane node as the monitoring and log disk.
   • Network Plugin: the network plugin used by the cluster, which is fixed to Calico.
   • Network Mode: select the network mode of the cluster. The default is VXLAN.
     • VXLAN (default): an overlay network. It has no requirement on the data center and is ready to use out of the box. It applies to most scenarios.
     • BGP: no encapsulation, and pod IP addresses can be accessed directly from outside the cluster. It applies to the scenarios where BGP is already used in the data center, where pod IP addresses need to be accessed directly from outside the cluster, or where the performance requirement is extremely high.
     Note: Preconditions for selecting the BGP mode: the data center switches support BGP and have been configured with BGP (AS/neighbors), the pod network segment does not conflict with the data center network segments, and TCP port 179 is allowed. The BGP mode requires the data center network to support BGP and to have been configured for interconnection, otherwise the cluster network is not connected.
     Note: The BGP mode currently has the following usage limitations:

     • Only business clusters support the BGP mode. The management cluster does not support it.
     • A cluster in the BGP mode does not support enabling microservices governance or deploying the microservices governance component.
     • A cluster in the BGP mode does not support using the pod external network.
     • A cluster in the BGP mode does not support cluster upgrade currently.
   • Router BGP Address: this field must be entered when the network mode is BGP. Enter the BGP address of the peer router. It must be in IPv4 format and cannot conflict with the node IP addresses, the pod network segment or the service network segment.
   • Router AS Number: this field must be entered when the network mode is BGP. Enter the AS number of the peer router. It can only be an integer from 1 to 4294967294, and 23456 and 65535 cannot be entered.
   • Cluster AS Number: this field must be entered when the network mode is BGP. Enter the AS number used by the cluster. It can only be an integer from 1 to 4294967294, and 23456 and 65535 cannot be entered. It cannot be the same as the router AS number or the AS number of an existing cluster on the platform.
   • BGP RR Node: this field is displayed when the network mode is BGP. It refers to the nodes that establish BGP sessions with the peer router, and they must be configured as BGP neighbors on the router. For an HA cluster, it is fixed to the 3 control plane nodes entered in the node configuration. For a cluster with a single control plane node, it is fixed to this control plane node.
   • Pod Network Segment: enter the network segment used by the pods in the cluster, which is used to allocate the IP addresses used by the pods.
     • When the network mode is VXLAN, the default is 10.233.64.0/18. It must be in CIDR format and the mask must be less than 24. If the default pod network segment is modified, avoid overlapping with the intranet network segment and the other network segments used when this cluster is created.
     • When the network mode is BGP, there is no default value. It must be in CIDR format and the mask must be less than or equal to 26. This network segment is advertised to the physical network and is routable outside the cluster, so enter a network segment that is allocated by the network administrator, is globally unique and has been allowed on the router side. Meanwhile, avoid overlapping with the existing network segments in the data center, the network segment where the management network IP addresses of the nodes are located and the pod network segments of the other clusters.
   • Node IP Block Mask: select the mask of the IP blocks requested from the pod network segment each time for the pods on a node. /24 to /30 are available and the default is /26 (64 IP addresses are requested each time). The mask must be greater than the mask of the pod network segment. When the network segment is limited or the number of the pods on a single node is small, the mask can be increased to accommodate more nodes.
     Note: The node IP block mask cannot be modified after the cluster is created.
   • Service Network Segment: enter the network segment used by the services in the cluster, which is used to allocate the ClusterIP addresses used by the services. The default is 10.233.0.0/18. If the default service network segment is modified, avoid overlapping with the intranet network segment and the other network segments used when this cluster is created.
   • DNS Server: add DNS servers to provide DNS resolution services and resolve the domain names outside the cluster. If no dedicated DNS server is deployed, you can enter the address of a public DNS server, such as 223.5.5.5.
   • Microservices Governance: select whether to enable microservices governance. After it is enabled, the microservices governance related components are deployed in the cluster, and the microservices governance related features can be used in the cluster. It cannot be enabled when the network mode is BGP.
   • Image Data Disk: optional. Select the image data disk used to store the image data. If no image data disk is set, the image data is stored on the system disk by default. When there are many large images in use, it is recommended to attach a dedicated disk to the nodes to store the images, so as to avoid the image data filling up the system disk and causing system exceptions.
3. Configuration confirmation

   Confirm the configuration information of the cluster. You can go back to modify it.

   After you click OK, the system starts to create the cluster. You can click Cluster Management > Cluster Management > Cluster to enter the Cluster page and view the creation progress of the cluster.

Deploying the Microservices Governance Component

In the ZStack Zaku main menu, click Cluster Management > Cluster Management > Cluster to enter the Cluster page. Click the name of the cluster to enter the cluster details page, and click Functional Component to enter the functional component sub-page. Click the Deploy button after the microservices governance component in the functional component list, and the Deploy Microservices Governance Component dialog box is displayed.

In the dialog box for deploying the microservices governance component, enter or select the namespace where the microservices governance component is deployed, and click OK.

Note:

• The default namespace is istio-system. If a new namespace is entered, the namespace is created by default and the microservices governance component is deployed in it.
• Make sure that no istio related component has been deployed under the namespace entered, so as to avoid conflicts.
• If the deployment of the microservices governance component fails, it can be deployed again, and the namespace cannot be changed then.
• The microservices governance component cannot be deployed again in a cluster where it has been deployed successfully.
• A cluster whose network mode is BGP does not support microservices governance, and the microservices governance component cannot be deployed in it.

Preparations

Deploying ZStone-RBD-CSI

In the ZStack Zaku main menu, click Cluster Management > Cluster Management > Cluster to enter the Cluster page. Click the name of the cluster to enter the cluster details page, and click Functional Component to enter the functional component sub-page. Click the Deploy button after ZStone-RBD-CSI in the functional component list, and the Deploy ZStone-RBD-CSI dialog box is displayed.

Note:

• The CSI of ZStone and that of ZCE cannot be deployed in the same cluster at the same time.
• ZStone-RBD-CSI cannot be deployed again in a cluster where it has been deployed successfully.
• For a cluster where the deployment failed, it can be uninstalled and then deployed again.
• ZStone-RBD-CSI is deployed in the zstack-csi namespace by default. When the CSI is deployed, a storageclass named zaku-rbd is created at the same time, and PVCs can be created with this storageclass to use block storage volumes.
• You can view the detailed deployment logs on the operation log sub-page of the cluster details page.

Preparations

Deploying ZCE-iSCSI-CSI

In the ZStack Zaku main menu, click Cluster Management > Cluster Management > Cluster to enter the Cluster page. Click the name of the cluster to enter the cluster details page, and click Functional Component to enter the functional component sub-page. Click the Deploy button after ZCE-iSCSI-CSI in the functional component list, and the Deploy ZCE-iSCSI-CSI dialog box is displayed.

Note:

• The CSI of ZStone and that of ZCE cannot be deployed in the same cluster at the same time.
• ZCE-iSCSI-CSI cannot be deployed again in a cluster where it has been deployed successfully.
• For a cluster where the deployment failed, it can be uninstalled and then deployed again.
• ZCE-iSCSI-CSI is deployed in the zstack-csi namespace by default. When the CSI is deployed, a storageclass named zaku-rbd is created at the same time, and PVCs can be created with this storageclass to use block storage volumes.
• You can view the detailed deployment logs on the operation log sub-page of the cluster details page.

Preparations

Deploying ZCE-NFS-CSI

In the ZStack Zaku main menu, click Cluster Management > Cluster Management > Cluster to enter the Cluster page. Click the name of the cluster to enter the cluster details page, and click Functional Component to enter the functional component sub-page. Click the Deploy button after ZCE-NFS-CSI in the functional component list, and the Deploy ZCE-NFS-CSI dialog box is displayed.

Note:

• The CSI of ZStone and that of ZCE cannot be deployed in the same cluster at the same time.
• ZCE-NFS-CSI cannot be deployed again in a cluster where it has been deployed successfully.
• For a cluster where the deployment failed, it can be uninstalled and then deployed again.
• ZCE-NFS-CSI is deployed in the zstack-csi namespace by default. When the CSI is deployed, a storageclass named zaku-share is created at the same time, and PVCs can be created with this storageclass to use file storage.
• You can view the detailed deployment logs on the operation log sub-page of the cluster details page.

Deploying the Workload Auto-Scaling Component

In the ZStack Zaku main menu, click Cluster Management > Cluster Management > Cluster to enter the Cluster page. Click the name of the cluster to enter the cluster details page, and click Functional Component to enter the functional component sub-page. Click the Deploy button after the workload auto-scaling component in the functional component list, and the Deploy Workload Auto-Scaling Component dialog box is displayed.

After the confirmation, the platform performs a pre-check. After the check is passed, the platform starts to create and manage the system resources required by the workload auto-scaling component in the cluster, including the scaling controller, the metrics service, the admission validation service and the related cluster resources.

Note:

• Only self-built clusters support deploying the workload auto-scaling component. This component is not displayed in the functional component list of an integrated cluster.
• The component is deployed in the zstack-autoscaling namespace by default. This namespace is reserved by the system and is not visible on the namespace page or other pages.
• If the system detects that workload auto-scaling (KEDA) related resources that are not deployed by the platform already exist in the current cluster, the deployment cannot be performed, so as to avoid cluster-level resource conflicts caused by duplicate deployment. Clean up the related resources and try again.
• If the system detects that a zstack-autoscaling namespace that is not created by the platform already exists in the cluster, the deployment cannot be performed, so as to avoid deployment conflicts or an impact on the existing resources in this namespace. Migrate the resources in it, delete the namespace and then try again.
• After the component is deployed successfully, it cannot be uninstalled manually currently. When the deployment fails, it can be deployed again or uninstalled.

How It Works

Key Features

Typical Application Scenarios

Usage Restrictions

Preparations

Adding a Node

In the ZStack Zaku main menu, click Cluster Management > Cluster Management > Node to enter the Node page. Click Add Node to enter the Add Node page.

Adding a node includes the following three steps:

1. Node configuration
   You can refer to the following example to enter the required information:

   • Cluster Name: displays the name of the current cluster by default.
   • Reuse the Business Network as the Management Network: displays whether the business network is reused as the management network in the current cluster by default. If it is enabled, the management network IPv4 address of a node is the same as its business network IPv4 address.
   • Node Information: enter the information of the nodes to be added.
     • Adding Method: the method of adding nodes, including IP address and IP range.
     • Business Network IPv4 Address: enter the business network IPv4 address of the node. Enter this field when the adding method is IP address.
     • Management Network IPv4 Address: enter the management network IPv4 address of the node. Enter this field when the adding method is IP address. When the business network is reused as the management network, the management network IPv4 address is the same as the business network IPv4 address.
     • Business Network IPv4 Range: enter the business network IPv4 range of the nodes. Enter this field when the adding method is IP range.
     • Management Network IPv4 Range: enter the management network IPv4 range of the nodes. Enter this field when the adding method is IP range. When the business network is reused as the management network, the management network IPv4 range is the same as the business network IPv4 range.
       Note:

       • Make sure that the IP addresses entered are consistent with the actual configuration of the machines.
       • Make sure that the management network IPv4 addresses and the management cluster are connected. When the business network is reused as the management network, make sure that the business network IPv4 addresses and the management cluster are connected.
       • Make sure that the business network IPv4 addresses of the nodes are connected to each other.
       • When the business network is not reused as the management network, make sure that the IP addresses in the management network IPv4 range correspond to those in the business network IPv4 range one by one. The IP addresses in the same order belong to the same node. For example, the 1st IP address in the business network IPv4 range and the 1st IP address in the management network IPv4 range belong to the 1st node.
       • When nodes are added in batches, the management network IPv4 addresses of the nodes in this batch cannot be duplicated, and the business network IPv4 addresses of the nodes in this batch cannot be duplicated. When the business network is not reused as the management network, a management network IPv4 address and a business network IPv4 address cannot be duplicated.
       • The management network IPv4 addresses and the business network IPv4 addresses of the nodes in this batch cannot be the same as those of the existing nodes in the cluster.
     • Node Name: set the name of the node. Naming rules: the length is limited to 2 to 45 characters, and the input can contain only lowercase letters, digits and the separator (-). It must start with a lowercase letter and cannot end with the separator. When the adding method is IP range, a start suffix must be specified. The node name consists of the name and the suffix, and the suffix increases from the start suffix in turn, such as "-1/-2/-3".
       Note:

       • The hostname of a host is set to the node name during the process of adding the node.
       • A node name cannot be the same as that of an existing node in the cluster.
       • When nodes are added in batches, the names used by the nodes in this batch cannot be duplicated.
     • Node Role: the default is compute node, and GPU node is available.
       Note:

       • The corresponding components are installed automatically on a GPU node during the deployment process. Select whether to install a machine as a GPU node according to the actual situation.
       • If GPU node is selected, make sure that the GPU driver has been installed correctly on the node. If the driver is not installed, only the corresponding components are installed and the node is not set to the GPU node role.
     • GPU Manufacturer: this field must be entered when GPU node is selected as the node role. Select the manufacturer of the GPUs on the node, and the system installs the GPU-related components of the corresponding manufacturer. Currently the following manufacturers are supported: NVIDIA, Huawei, HYGON, Iluvatar and Alibaba
       Note:

       • The huawei NPU driver and related components require the system accounts HwHiAiUser and hwMindX, and UID/GID 1000 and 9000. A UID/GID conflict message means that UID/GID 1000 or 9000 on the listed nodes is already used by users or groups other than these huawei accounts.
       • If huawei is selected as the GPU manufacturer, resolve the conflicts on the listed nodes and run the check again. You can continue only after the check passes.
       • If huawei is not selected as the GPU manufacturer, you can acknowledge the risk and continue. However, resolve the conflicts before using huawei NPUs later, or the driver or related components may fail to install.
   • Iluvatar GPU Virtualization License: this field must be entered when there is no Iluvatar GPU in the cluster yet and there is a GPU node whose GPU manufacturer is Iluvatar in this addition. Enter the license of Iluvatar GPU virtualization. This license is required to share Iluvatar GPUs in containers, and it is obtained from Iluvatar officially.
   • Node ROOT Password: enter the password of the root account of the nodes, which is used to connect to the nodes over SSH to execute the installation steps during the process of adding the nodes. Make sure that the ROOT password of each node is the same.
   • Node SSH Port: enter the port used to connect to the nodes over SSH, which is used to connect to the nodes over SSH to execute the installation steps during the process of adding the nodes. Make sure that this port of each node can be connected.
   • DNS Server: add DNS servers to provide DNS resolution services and resolve the domain names outside the cluster. If no dedicated DNS server is deployed, you can enter the address of a public DNS server, such as 223.5.5.5.
2. Advanced configuration
   You can refer to the following example to enter the required information:

   • Image Data Disk: optional. Select the image data disk used to store the image data. If no image data disk is set, the image data is stored on the system disk by default. When there are many large images in use, it is recommended to attach a dedicated disk to the node to store the images, so as to avoid the image data filling up the system disk and causing system exceptions.
3. Configuration confirmation

   Confirm the configuration information of the nodes. You can go back to modify it.

   After you click OK, the system starts to add the nodes and the cluster status changes to Scaling. You can click Cluster Management > Cluster Management > Cluster to enter the Cluster page and view the progress of adding the nodes.

Note:

• Currently only compute nodes and GPU nodes can be added. Control plane nodes cannot be added.
• Nodes cannot be added when the cluster is in the Abnormal or Deleting state.

Viewing the GPU Pool

In the ZStack Zaku main menu, click Cluster Management > Cluster Management > GPU Pool to enter the GPU Pool page.

Managing GPUs

GPUs support the following operations:

Operation	Description
GPU Details	Click a GPU UUID to view its basic information, hardware information, real-time load, and scheduled pods. Monitoring charts are available for GPU utilization, GPU memory utilization, temperature, power consumption, fan speed, and PCIe transmit and receive throughput. Note: Available monitoring metrics depend on the GPU manufacturer and model. You can still query monitoring data collected before a device entered the Crashed or Dropout state.
Remove	Remove the record of a GPU in the Dropout state from the GPU pool. Note: Only GPUs in the Dropout state can be removed. A recovered device appears in the GPU pool again.