Deployment Planning
Term Definitions
| Term | Definition |
|---|---|
| management cluster | The environment that hosts the container service platform and its components. It consists of management cluster nodes. A dedicated management cluster does not host customer workloads. Starting with version 3.10.0, it can also host customer workloads when reused as a business cluster. |
| management cluster node | A physical server or VM instance on which the container service platform is deployed. |
| basic service VIP | An IP address that provides high availability for internal service components in the management cluster. |
| console VIP | An IP address through which the management cluster exposes the web console and the upload and download endpoints of the image repository. |
| system data disk | A data disk used to store platform component data in the management cluster. Each management cluster node must have a dedicated system data disk. |
| business cluster | A Kubernetes cluster created through the container service platform to host customer workloads. It has its own control planes and compute nodes. |
| managed cluster | A third-party Kubernetes cluster that can be integrated into the container service platform by using a valid kubeconfig file. |
| control plane | The node role referred to as a control plane node in Kubernetes documentation. It acts as the brain of a business cluster, making decisions and scheduling and managing Kubernetes resources. This role does not directly host customer workloads; the same node can also serve as a compute node in the shared-role deployment options described below. |
| compute node | A node that runs customer workloads, referred to as a worker node in Kubernetes documentation. |
| management network | The network plane through which a business cluster communicates with the management cluster. It is primarily used for cluster management, image distribution, and monitoring/logging queries. |
| business network | The network plane that carries communication between containers running customer workloads in a business cluster. |
| business network VIP; management network VIP | IP addresses through which a business cluster exposes components such as kube-apiserver, monitoring endpoints, and logging endpoints. |
| monitoring/logging data disk | A data disk used to store monitoring data and container logs in a business cluster. Each control plane must have a dedicated monitoring/logging data disk. |
| container images disk | A disk used to store data for container runtimes such as containerd in a business cluster. When many or large images are used, attach a dedicated disk to each node to store images and prevent system failures caused by a full system disk. |
| pod ip range | The range from which pod IP addresses are allocated in a business cluster. It is used only within the cluster and is not routed outside the cluster. |
| service CIDR | The range from which virtual IP addresses (ClusterIP) are allocated to services in a business cluster. It is used only within the cluster and is not routed outside the cluster. |
| external network | A separate IP address range comprising a Pod External Network and a Service External Network. |
| Pod External Network | A network that provides dedicated IP addresses for pods in a business cluster. These IP addresses can be accessed directly by physical servers, VM instances, and other clients outside the business cluster. |
| Service External Network | A network that provides dedicated IP addresses for services in a business cluster. These IP addresses can be accessed directly by physical servers, VM instances, and other clients outside the business cluster. |
Deployment Workflow
Review the deployment architecture, hardware compatibility, and license metering methods.
Review node planning, network planning, and performance requirements.
Complete the platform information collection tables before deployment.
Download the installation packages, install the operating system, and configure networking.
Install the management platform.
Create a business cluster.
Install optional components.
Perform platform inspection and functional tests.
Deployment Architecture
ZStack Zaku Container Service includes a management cluster, business clusters, and managed clusters. Before deployment, read this section to select an appropriate deployment architecture and plan node capacity and cluster size.
Management Cluster
The management cluster runs the management components of ZStack Zaku Container Service. Each ZStack Zaku Container Service deployment contains one management cluster.
Starting with version 3.10.0, the management cluster can also host customer workloads.
The management cluster supports two deployment options:
| Architecture | Number of nodes | Characteristics |
|---|---|---|
| Non-HA architecture | 1 + N; 1: number of control planes; N: number of compute nodes | Uses fewer resources and supports demonstrations of all features. The platform has no overall high availability. Use it for demonstrations or POC functional testing, and avoid using it in production. Control planes can also serve as compute nodes in an All in One deployment to reduce resource consumption. |
| HA architecture | 3 + N; 3: number of control planes; N: number of compute nodes | Uses more resources and provides full functionality. Suitable for production. Control planes can also serve as compute nodes to reduce resource consumption, but this arrangement is not recommended and should be used only when customer resources are insufficient. |
Business Cluster
A business cluster is a Kubernetes cluster created and managed throughout its lifecycle by the container service platform to host customer workloads. A single platform deployment can create and manage multiple business clusters.
| Architecture | Number of nodes | Characteristics |
|---|---|---|
| Non-HA architecture | 1 + N; 1: number of control planes; N: number of compute nodes | Uses fewer resources and supports demonstrations of all features. The platform has no overall high availability. Use it for demonstrations or POC functional testing, and avoid using it in production. Control planes can also serve as compute nodes to reduce resource consumption. |
| HA architecture | 3 + N; 3: number of control planes; N: number of compute nodes | Uses more resources and provides full functionality. Suitable for production. At least 2 compute nodes are recommended to provide a minimum level of redundancy for compute node high availability. Control planes can also serve as compute nodes to reduce resource consumption. |
Managed Cluster
A managed cluster is a standard Kubernetes cluster created by a third-party provider. Upload its kubeconfig file to the container service platform to manage its resources. Integrating an existing cluster avoids workload migration and the associated troubleshooting and interruptions, but has the following limitations:
Kubernetes cluster maintenance is not provided, and the stability and high availability of managed clusters are not guaranteed. Node tuning and operations such as adding or removing nodes are not provided.
Some features are unavailable, including microservices governance, monitoring/logging alerts, and external networks.
Hardware Compatibility
ZStack Zaku Container Service supports deployment with loosely coupled software and hardware. It can run on physical servers and VM instances with x86 or ARM architectures, and supports mixed deployments of physical servers and VM instances as well as mixed x86 and ARM nodes. Prepare an appropriate node configuration list based on actual requirements. The platform imposes no restrictions or special requirements on hardware brands. Existing servers can be reused and managed centrally through the container service platform.
ZStack Zaku Container Service can also be integrated with ZStack Cloud and ZStack Enterprise Distributed Storage to form a complete solution. ZStack Cloud provides unified licensing, unified management of VM instances and containers, and automatic scaling of business cluster compute nodes. Block and file storage from ZStack Enterprise Distributed Storage provides persistent storage for business cluster data.
License Metering
Which nodes count toward license usage?
Nodes in ZStack Zaku Container Service are generally divided into Kubernetes control planes and compute nodes. Their treatment for license metering is as follows:
| node role | License metering |
|---|---|
| control plane | Excluded from license metering |
| compute node | Included in license metering |
How is compute node license usage calculated?
As shown below, ZStack Zaku Container Service provides two flexible license metering modes, vCPU authorization and CPU authorization, to meet different workload requirements:
vCPU authorization: Used when compute nodes are VM instances. Usage is metered by the number of logical processors (threads) in each VM instance.
CPU authorization: Used when compute nodes are physical servers. Usage is metered by the number of processors (sockets) in each physical server.
Node Sizing
Minimal Deployment (All in One)
When customer resources are limited to one or a few nodes for POC testing, a single-node All in One deployment can maximize resource utilization. The same node hosts both management platform components and customer workloads.
A management cluster with at least one node is intended for minimal platform deployment and functional testing. It is not suitable for scenarios involving isolation across multiple networks, performance testing, or HA testing.
| Cluster type | Node type | Quantity | Node specifications |
|---|---|---|---|
| management cluster (also used as a business cluster) | management cluster node | 1 | 16 CPU cores, 32 GB memory; 300 GB system disk + 200 GB data disk (hybrid flash or better); 1 Gbps NIC * 1 |
Production Deployment with Physical Servers
If the customer has only a few physical servers and cannot provide VM instances, a separate management cluster plus business cluster deployment is not recommended because a dedicated management cluster would leave physical server resources idle or underused.
This option requires at least three physical servers to serve as both control planes and compute nodes in the management cluster. Add the remaining physical servers as compute nodes in the same cluster to fully utilize their compute resources.
A management cluster with at least three nodes provides high availability. This option is suitable for production deployment and transition from POC testing to production when there are no special storage integration or network isolation requirements.
| Cluster type | Node type | Quantity | Node specifications |
|---|---|---|---|
| management cluster (also used as a business cluster) | control plane; also serves as a compute node | 3 | 16 CPU cores, 32 GB memory; 500 GB system disk + 500 GB data disk (hybrid flash or better); 10 Gbps NIC * 1 |
| management cluster (also used as a business cluster) | compute node | 0 or more | 16 CPU cores, 32 GB memory; 500 GB system disk (hybrid flash or better); 10 Gbps NIC * 1 |
Standard Production Deployment with VM Instances
The customer provides several VM instances for the container service platform, with no physical servers available.
The management cluster consists of three VM instances and does not run customer workloads, preventing those workloads from affecting platform stability.
(Recommended) The business cluster consists of control planes and compute nodes. Three VM instances serve as control planes and do not run customer workloads, preventing those workloads from affecting cluster stability. Add the remaining VM instances as compute nodes to run customer workloads.
(Alternative) The business cluster consists of control planes and compute nodes. When VM instance resources are limited, three VM instances serve as both control planes and compute nodes to run customer workloads. Add the remaining VM instances as compute nodes to run customer workloads.
An HA deployment comprising a management cluster (three nodes) and a business cluster (at least three nodes) is suitable for production deployment and transition from POC testing to production when there are no special storage integration or network isolation requirements.
| Cluster type | Node type | Quantity | Node specifications |
|---|---|---|---|
| management cluster | management cluster node | 3 | 16 CPU cores, 32 GB memory; 500 GB system disk + 500 GB data disk (hybrid flash or better); 10 Gbps NIC * 1 |
| Kubernetes business cluster | control plane; also serves as a compute node | 3 | 16 CPU cores, 32 GB memory; 500 GB system disk + 500 GB data disk (hybrid flash or better); 10 Gbps NIC * 1 |
| Kubernetes business cluster | compute node | 0 or more | 16 CPU cores, 32 GB memory; 500 GB system disk (hybrid flash or better); 10 Gbps NIC * 1 |
Standard Production Deployment with VM Instances and Physical Servers
The customer provides both VM instances and physical servers for the container service platform. Use VM instances as control planes and physical servers as compute nodes to maximize the utilization of physical server compute resources.
The management cluster consists of three VM instances and does not run customer workloads, preventing those workloads from affecting platform stability.
The business cluster consists of control planes and compute nodes. Three VM instances serve as control planes and do not run customer workloads, preventing those workloads from affecting cluster stability. Add the remaining VM instances or physical servers as compute nodes to run customer workloads.
An HA deployment comprising a management cluster (three nodes) and a business cluster (at least three nodes) is suitable for production deployment and transition from POC testing to production when there are no special storage integration or network isolation requirements.
| Cluster type | Node type | Quantity | Node specifications |
|---|---|---|---|
| management cluster | management cluster node | 3 | 16 CPU cores, 32 GB memory; 500 GB system disk + 500 GB data disk (hybrid flash or better); 10 Gbps NIC * 1 |
| Kubernetes business cluster | control plane | 3 | 16 CPU cores, 32 GB memory; 500 GB system disk + 500 GB data disk (hybrid flash or better); 10 Gbps NIC * 1 |
| Kubernetes business cluster | compute node | 2 or more | 16 CPU cores, 32 GB memory; 500 GB system disk (hybrid flash or better); 10 Gbps NIC * 1 |
General Planning
Nodes in the container service platform are generally classified as management cluster control planes, business cluster control planes, and compute nodes in either cluster type. This section provides minimum node resource requirements for POC and production environments. Increase the resources as needed for actual workloads.
Management Cluster Node Resource Requirements
Physical servers or VM instances used as management cluster nodes must meet the following hardware requirements:
| Hardware | Minimum (POC environment) | Recommended (production environment) |
|---|---|---|
| Host | Physical server or VM instance | Physical server or VM instance |
| Quantity | 1 | 3 |
| CPU | 16 cores (can be reduced to 8 cores for a 3-node deployment) | 16 cores or more |
| Memory | 32 GB (can be reduced to 16 GB for a 3-node deployment) | 32 GB or more |
| System disk | 300 GB | 500 GB or more |
| data disk | 200 GB (must be increased to 400 GB for a 3-node deployment) | 500 GB or more |
| container images disk (optional) | 100 GB | 300 GB or more |
| NIC | 1 Gbps * 1 | 10 Gbps * 1 |
| Network | Connectivity between nodes | Connectivity between nodes |
Business Cluster Control Plane Resource Requirements
Business cluster control planes must meet the following hardware requirements:
| Hardware | Minimum (POC environment) | Recommended (production environment) |
|---|---|---|
| Host | Physical server or VM instance | Physical server or VM instance |
| Quantity | 1 | 3 |
| CPU | 8 cores | 16 cores or more |
| Memory | 16 GB | 32 GB or more |
| System disk | 300 GB | 500 GB or more |
| data disk | 400 GB | 500 GB |
| container images disk (optional) | 100 GB | 200 GB or more |
| NIC | 1 Gbps * 1 | 10 Gbps * 1 |
| Network | Connectivity between nodes | Connectivity between nodes |
Compute Node Resource Requirements
Business cluster compute nodes must meet the following hardware requirements:
| Hardware | Minimum | Recommended (production environment) |
|---|---|---|
| Host | Physical server or VM instance | Physical server or VM instance |
| Quantity | 1 | 2 or more |
| CPU | 4 cores | 16 cores or more |
| Memory | 8 GB | 32 GB or more |
| System disk | 100 GB | 500 GB or more |
| container images disk (optional) | 100 GB | 200 GB or more |
| NIC | 1 Gbps * 1 | 10 Gbps * 1 |
| Network | IP connectivity to other host nodes | IP connectivity to other host nodes |
Storage Capacity Planning
As described in Node Sizing, management cluster nodes and business cluster control planes require data disks, separate from system disks and container images disks, to store additional persistent files. These data disks store platform database files, monitoring data, log data, container image files, and other data.
The data disk capacities listed above generally support 7 days of data retention for a small cluster with fewer than 10 nodes. They may not meet customer requirements for monitoring/logging retention or image repository capacity. Evaluate the required data disk capacity accordingly.
For deployments on physical servers, thoroughly evaluate data disk capacity requirements.
Estimating Monitoring Storage Capacity
As a reference, monitoring 130 pods with a retention period of 30 days uses approximately 25Gi of storage.
| cluster | namespace | Component name | Storage required per control plane | Purpose |
|---|---|---|---|---|
| management cluster | monitoring | prometheus | 30Gi | Node and container monitoring data |
| business cluster | monitoring | prometheus | 100Gi | Node and container monitoring data |
Estimating Logging Storage Capacity
Logging storage capacity depends on the volume of logs produced by pods. There is no standard or reliable calculation formula. Estimate the requirements based on the log volume of the customer’s existing workloads.
| cluster | namespace | Component name | Storage required per control plane | Purpose |
|---|---|---|---|---|
| management cluster | monitoring | loki | 30Gi | Container logs, events, and audit records |
| business cluster | monitoring | loki | 250Gi | Container logs and events |
Estimating Image Repository Capacity
Image repository capacity depends on how the customer uses it. There is no standard or reliable calculation formula. Estimate the requirements based on existing workload image sizes, workload scale, and the container image update cycle.
| cluster | namespace | Component name | Storage required per control plane | Purpose |
|---|---|---|---|---|
| management cluster | zstack-storage | zstack-pool | 300Gi | Image repository backend storage; 9 data disks with EC:4 redundancy provide approximately 500Gi of usable capacity |
| business cluster | None | None | 0Gi | None |
Other Storage Allocations
The storage allocated to other platform components generally does not need to be adjusted. Retain the default values.
| cluster | Approximate storage required per control plane |
|---|---|
| management cluster | 30Gi |
| business cluster | 2Gi |
Storage requirement details:
| cluster | namespace | Component | Default capacity | Purpose |
|---|---|---|---|---|
| management cluster | zstack-middleware | postgres | 10Gi | Container service platform metadata |
| management cluster | zstack-middleware | redis | 5Gi | Platform login cache, image metadata cache, and other cached data |
| management cluster | monitoring | alertmanager | 10Gi | Alert rules |
| management cluster | zstack-devops | mongodb | 10Gi | DevOps module database |
| management cluster | zstack-devops | mysql | 5Gi | DevOps module database |
| management cluster | harbor | trivy | 3Gi | Vulnerability database cache |
| management cluster | harbor | jobservice | 1Gi | Job logs and other data |
| management cluster | harbor | jobservice-scandata | 1Gi | Raw scan report data |
| business cluster | monitoring | alertmanager | 2Gi | Alert rules |
Capacity Estimate Summary
| Capacity item | management cluster | business cluster |
|---|---|---|
| Monitoring capacity | 30Gi | 100Gi |
| Logging capacity | 30Gi | 250Gi |
| Image repository capacity | 300Gi | 0Gi |
| Other storage capacity | 30Gi | 2Gi |
| Total | ≈ 390Gi | ≈ 352Gi |
Network Planning
Network planning for the container service platform generally covers three logical networks: the management network, business network, and storage network. If customer applications are allowed to run in the management cluster, also follow the business cluster network architecture when planning its networks.
| Scope | Network name | Purpose |
|---|---|---|
| management cluster | management network | Communication between management cluster components and user access |
| business cluster | management network | Communication between the management cluster and business cluster; management of business clusters and nodes; image distribution; monitoring/logging queries |
| business cluster | business network | Autonomous management of Kubernetes components within the business cluster, container communication across nodes, and outbound container workload traffic using SNAT |
| business cluster | Storage network | Connects the business cluster to a storage platform to provide persistent storage for pvcs. Its design depends on the storage platform’s network architecture. |
Management and Business Network Planning
Before deploying management clusters and business clusters, review their management and business network architecture. The following two designs are available.
Shared Management and Business Network
Sharing the management network and business network is a common and simple network planning option.
In this design, each node has one NIC. The management cluster and business cluster can reside on different subnets and communicate through Layer 3 routing.
| Network architecture | management cluster | business cluster |
|---|---|---|
| Shared management/business network | N + 2 IP addresses; N: total nodes in the cluster; 2: basic service VIP + console VIP | N + 1 IP addresses; N: total nodes in the cluster; 1: business network VIP |
IP address requirements
Separate Management and Business Networks
Separating the management network and business network is intended for environments with higher network security requirements. The main difference is that management and business traffic use two independent networks.
The management networks of the management cluster and business cluster must be on the same subnet. Configure the default route of business cluster nodes on the business network.
| Network architecture | management cluster | business cluster |
|---|---|---|
| management network | N + 2 IP addresses; N: total nodes in the cluster; 2: basic service VIP + console VIP | N + 1 IP addresses; N: total nodes in the cluster; 1: management network VIP |
| business network | 0 | N + 1 IP addresses; N: total nodes in the cluster; 1: business network VIP |
IP address requirements
Storage Network Planning
ZStack Enterprise Distributed Storage provides block and file storage to store and retain critical workload data securely. It offers a highly available, high-performance persistent storage foundation and integrates with container workloads in business clusters to deliver cloud-native infrastructure spanning compute and storage.
Storage network designs vary by storage platform. The following designs use ZStack Enterprise Distributed Storage as examples:
Note: The management cluster does not need to integrate with distributed storage, so its topology is omitted from this section.
Applicable Storage: ZCE-X (iSCSI) and ZStone (RBD)
Add a storage network to all compute nodes in the business cluster.
Storage network requirements:
The management networks of the business cluster and storage cluster must be able to communicate.
Layer 2 connectivity is required between the business cluster and the storage cluster’s public storage network (public-network).
Note: A gateway is usually not configured for the public storage network, so Layer 2 connectivity is required.
Applicable Storage: ZStone (ZBS)
Add a storage network to all compute nodes in the business cluster.
Storage network requirements:
- Layer 2 connectivity is required between the business cluster and the storage cluster’s public storage network (public-network).
Note: A gateway is usually not configured for the public storage network, so Layer 2 connectivity is required.
Applicable Storage: ZCE-X (NFS)
Add a storage network to business cluster compute nodes as needed.
Storage network requirements:
- The business cluster must be able to communicate with the storage cluster’s NFS gateway address.
Network Planning When Used as the ZStack Cloud Container Module
In addition to the requirements in Management and Business Network Planning, ensure connectivity between the ZStack Cloud management network and the Zaku management cluster network.
Shared Management and Business Network Design
Ensure that both ZStack Cloud management nodes (MNs) can communicate with the management cluster.
Separate Management and Business Network Design
Both ZStack Cloud MNs must be on the same subnet as the management networks of the management cluster and business cluster.
Note: The ZStack Cloud management network typically uses 1 Gbps networking. For this design, upgrading it to 10 Gbps is recommended.
Performance Requirements
Disk requirements:
Management cluster nodes and business cluster control planes run etcd. Use high-speed storage, such as hybrid flash, SSD, or NVMe, to ensure etcd performance.
IOPS requirements:
| Item | Requirement |
|---|---|
| Random Write IOPS | The disk on each etcd node should provide at least 1000 to 2000 random write IOPS. |
| Random Read IOPS | The disk on each etcd node should provide at least 1000 to 2000 random write IOPS. |
| Sequential Write IOPS | Although etcd primarily performs random writes, sequential write IOPS must also be considered. The disk on each etcd node should provide at least 100 to 500 sequential write IOPS. |
| fsync latency | Synchronous data writes to disk must have latency below 10 ms to ensure etcd stability. |
Note: Local HDDs generally do not meet these performance requirements.
Network performance requirements:
| Item | Requirement |
|---|---|
| Bandwidth | At least 1 Gbps |
| Latency | Less than 0.5 ms to ensure high performance |
Platform Information Collection
After reviewing the deployment architecture and completing deployment planning, collect the information required for installation, configuration, and use in the following tables. Prepare this information before deployment.
When deploying on an IaaS virtualization platform, allocate VIP addresses appropriately to avoid IP conflicts. For example, create virtual IP resources in ZStack Cloud to reserve the basic service VIP and console VIP, preventing them from being assigned to other VM instances.
Installation package information
| Item | Example value | Notes |
|---|---|---|
| Product version | V3.10.0 | Main installation package version |
| DevOps add-on module | Yes | Install the DevOps add-on module |
| Storage plugin package | Yes | Install the CSI plugin for integration with ZStone-rbd, ZCE-iscsi, and ZCE-nfs |
Operating system information
| Item | Example value | Notes |
|---|---|---|
| CPU Architecture | x86_64 | x86_64 or arm64 |
| Operating system image | ZStack H84r | x86_64 supports H84r and Kylin V10SP3.2403; arm64 supports Kylin V10SP3.2403 |
| Operating system installation method | ISO | ISO or qcow2 |
| Operating system partitioning and precautions | Standard partitions; disable swap | For other precautions, see Installing the Operating System |
| node root password | password | Default qcow2 image password: zstack@123@% |
| DNS Server | 172.27.8.5 |
Optional |
| NTP server | 172.27.8.5 |
Optional |
ZStack Cloud integration (optional)
| Item | Example value | Notes |
|---|---|---|
| Cloud API address | Cloud API address | Licensing changes from standalone container service licensing to ZStack Cloud module licensing, with Container Management-CPU and Container Management-vCPU options |
| Cloud admin password | password | |
| Embed the container console in ZStack Cloud | Yes | Embed the console pages in ZStack Cloud |
Management cluster network configuration
| Item | management network information | Notes |
|---|---|---|
| IP addresses | 10.16.209.21; 10.16.209.22; 10.16.209.23 |
|
| Netmask | 255.255.255.0 |
|
| Gateway | 10.16.209.254 |
|
| VLAN_ID | trunk + VLAN_ID=9 | access or trunk+VLAN_ID=10 |
| basic service VIP | 10.16.209.20 |
Must be on the same subnet as the node IP addresses |
| console VIP | 10.16.209.25 |
Must be on the same subnet as the node IP addresses |
Business cluster-1 information
| Item | Example value | Notes |
|---|---|---|
| Cluster Name | cluster-1 | A combination of letters, hyphens (“-”), and digits |
| Kubernetes version | 1.30 | 1.24 or 1.30 |
| Number of control planes | 3 | A single-node cluster has 1 control plane (POC only; not for production). An HA cluster has 3 control planes. |
| Number of compute nodes | 5 | At least 2 are recommended for an HA cluster; included in license metering |
Business cluster network configuration
| Network configuration | IPMI information | management network information | business network information | Storage network information |
|---|---|---|---|---|
| IP addresses | 10.16.200.21; 10.16.200.22; 10.16.200.23 |
10.16.209.41; 10.16.209.42; 10.16.209.43 |
10.16.210.41; 10.16.210.42; 10.16.210.43 |
192.168.209.41; 192.168.209.42; 192.168.209.43 |
| Netmask | 255.255.255.0 |
255.255.255.0 |
255.255.255.0 |
255.255.255.0 |
| Gateway | 10.16.200.254 |
10.16.209.254 |
10.16.210.254 |
None |
| Cluster VIP | None | 10.16.209.45 |
10.160.210.45 |
None |
| VLAN_ID | access | access | trunk VLAN_ID=10 | access |
| Login credentials | admin / password | root / password | root / password | root / password |
| pod ip range | None | 10.233.64.0/18 |
10.233.64.0/18 |
10.233.64.0/18 |
| service CIDR | None | 10.233.0.0/18 |
10.233.0.0/18 |
10.233.0.0/18 |
| Service External Network | None | None | 10.16.210.30~79 |
None |
| Pod External Network | None | None | 10.16.210.80~89 |
None |