ZCF supports container service capabilities that extend the cloud platform's compute, network, and storage capabilities to container clusters and cloud-native applications, providing cluster management, resource scheduling, application orchestration, artifact delivery, O&M governance, and security control for containerized workloads. Container service capabilities are provided by ZStack Zaku.

Technical Architecture

The container service adopts a combined architecture comprising management clusters, workload clusters, and managed clusters. The management cluster hosts container service management components; workload clusters host containerized applications created and managed through the container service; and managed clusters integrate existing Kubernetes clusters into a unified management view.

Management Cluster

The management cluster hosts the container service's management components. Each container service deployment includes exactly one management cluster. After deployment, administrators can use the management console to centrally manage workload clusters, managed clusters, images, application packages, workloads, and O&M data, providing a consistent operational entry point for administrators, operators, and auditors.

The container service mainly supports managing the following resources:

• Cluster: A group of compute nodes (hosts or VM instances).
• Node: A node (host or VM instance) that provides compute, network, and storage resources for Pod instances.
• Local registry: A storage server used to store container images and application packages.
• Container image: A template file for a container.
• Application: A Helm application package—a collection of Kubernetes resources.
• Workload: Defines the desired operational state of Pods, such as Pod configuration, replica count, and scheduling policies.
• Pod: A collection of running container instances.
• Service: Provides service discovery and Layer-4 load-balanced ingress for a set of Pods.
• Ingress: Provides Layer-7 load-balanced ingress for a set of Pods.
• Network policy: A policy that controls inbound and outbound network traffic for Pods.
• Volume: Used for persistent storage of container data, ensuring data retention across container restarts, migrations, or deletions.
• Configuration Sets and Secret Dictionaries: Store configuration data as key-value pairs and inject them into containers, decoupling container images from configuration information.

Container Service organizes management capabilities across three cluster types: Management Clusters, Business Clusters, and Managed Clusters. The Management Cluster hosts Container Service control-plane components and provides cluster provisioning, resource management, artifact management, and operations capabilities. Business Clusters—provisioned by the Management Cluster—run containerized workloads. Managed Clusters integrate existing Kubernetes clusters into the unified management view, enabling centralized oversight of pre-existing container environments.

Within a Business Cluster, Nodes provide compute, network, storage, and device resources. Kubernetes abstracts these resources into a schedulable container runtime environment. Subsequent sections elaborate on compute resource isolation, scheduling policies, and device usage models—specifically at the boundaries of Business and Managed Clusters.

A workload cluster is a Kubernetes cluster created and managed by a management cluster to host containerized applications. A single management cluster can manage multiple workload clusters, enabling isolated execution of different workloads, tenants, or environments—and unified resource management, application delivery, and operations through a common entry point.

Workload clusters come pre-installed with monitoring, log and event collection, operational auditing, and service governance components, providing continuous runtime observability, issue diagnosis, and governance capabilities after application deployment—reducing the complexity for users to manually assemble foundational container components.

Workload clusters support the following key capabilities:

• Workload Cluster Lifecycle Management
• Manual and automatic node scaling
• Standard Kubernetes resource management
• Heterogeneous CPU architecture and heterogeneous GPU device management
• Support for multiple Kubernetes distributions
• Support for CNI plugins including Calico VXLAN, Calico BGP, Pod external networking, and Service external networking
• Integration with distributed storage solutions such as ZStack ZStone, ZStack ZBS, and ZStack ZCE-X
• Provides monitoring and alerting capabilities.
• Provides log and event collection capabilities.
• Provides microservice governance plugins.

A managed cluster is a standard Kubernetes cluster created by a third-party platform or external environment. After the administrator uploads the kubeconfig file, the cluster is integrated into the Unified Portal for centralized visibility and management—without requiring application migration.

Managed clusters are suitable for integrating existing container environments, reducing risks associated with application migration, reconfiguration, and service disruption. However, because the cluster is neither created nor maintained by Container Service, certain cluster-level maintenance and enhancement capabilities are limited:

• Kubernetes cluster-level maintenance is not provided; cluster stability and high availability are not guaranteed; node optimization, node addition, and node removal operations are not supported.
• Some features are unavailable, such as microservice governance, monitoring/log/alerting, and external networking.

Compute virtualization in Container Service targets containerized workloads, abstracting CPU, memory, and device resources from Nodes into requestable, schedulable, and isolatable runtime resources. Unlike hardware-level virtualization (e.g., KVM), Kubernetes relies primarily on Linux Namespaces and Control Groups (cgroups) at the OS level to deliver resource isolation and capacity enforcement.

Namespaces isolate process trees, network stacks, mount points, user IDs, and hostnames; cgroups limit and monitor resource consumption (e.g., CPU, memory). Leveraging these mechanisms, containers achieve relatively isolated execution environments while sharing the host kernel—and the scheduler selects optimal Nodes based on resource requests, limits, and Node conditions.

CPU virtualization governs how containerized workloads consume CPU resources on a Node, ensuring stable co-location of multiple Pods on shared infrastructure. Kubernetes translates Pod CPU requests and limits into kernel scheduling parameters using cgroups and the CPU subsystem.

From the host kernel's perspective, container processes remain ordinary schedulable entities—without dedicated CPU privilege levels. CPU requests guide scheduling decisions (e.g., Node selection and resource reservation); CPU limits enforce time-slice constraints during runtime. When multiple Pods compete for CPU, the kernel scheduler allocates CPU time proportionally per these parameters—preventing any single Pod from monopolizing excessive compute resources.

Memory virtualization establishes reservable and enforceable memory boundaries for Pods. Kubernetes leverages the Linux memory management subsystem and cgroups to track memory usage and constrain runtime memory allocations.

A container instance may specify memory requests and memory limits. Memory requests drive Node selection and resource reservation during scheduling—determining where the Pod is placed. Memory limits enforce capacity control at runtime. If a container's processes attempt to allocate more memory than its limit, the host kernel attempts memory reclamation; if insufficient memory remains after reclamation, the kernel triggers an Out-of-Memory (OOM) event inside the container—preventing abnormal memory consumption from impacting the Node or other workloads.

Device virtualization integrates heterogeneous devices (e.g., GPUs) into the container scheduling framework—enabling AI inference, model training, and graphics processing workloads to leverage dedicated hardware resources in containerized environments. Containers themselves do not provide hardware emulation (unlike KVM); instead, containerized processes typically access physical or virtualized devices via device nodes exposed by the host.

Kubernetes uses the Device Plugin framework to handle device discovery, resource reporting, and scheduler integration. Device plugins run on nodes and communicate with kubelet via gRPC, registering hardware capabilities—such as GPUs—as extended Kubernetes resources. After registration, users can request these device resources in Pod specifications, and the scheduler selects appropriate nodes based on node resource capacity, resource requests, and scheduling policies.

Container network virtualization provides intra-cluster communication, service discovery, access control, and external access for containerized applications. Based on the Kubernetes networking model and CNI plugins, Container Service organizes node network resources into container networks, service networks, network policies, and external networks—ensuring stable network connectivity for Pods across creation, scaling, migration, and recreation.

By default, container business clusters use Calico in VXLAN mode as the container network plugin. VXLAN is an overlay networking solution: when Pods communicate, their packets are encapsulated into VXLAN frames and transmitted between nodes over UDP port 4789. This mode imposes minimal requirements on the underlying network—it does not require BGP support from switches or complex routing configurations, and operates as long as nodes are IP-reachable. It is ideal for environments with constrained network conditions or where rapid deployment is required.

In VXLAN mode, Calico's network policy capabilities remain fully functional, allowing users to enforce fine-grained access control via Kubernetes NetworkPolicy objects. For environments where the underlying network supports BGP and higher network performance is required, Container Service supports switching to Calico's BGP routing mode. In BGP mode, nodes advertise their Pod CIDR ranges to the underlying network or other nodes using the BGP protocol and route reflectors, enabling native IP forwarding of Pod traffic—eliminating tunneling overhead and facilitating seamless integration with existing enterprise L3 networks.

In Kubernetes, a Pod's lifecycle is ephemeral. When a Pod is recreated due to rolling updates, node failures, or resource reclamation, its IP address changes. If clients access applications directly via Pod IPs, they must dynamically track these changes—an unsustainable approach in distributed systems. A core capability of Services is to provide a stable network endpoint for a dynamic set of backend Pods.

When a user creates a service, Kubernetes allocates a virtual IP address—called a ClusterIP—from the cluster's preconfigured service CIDR (typically a private subnet such as 10.233.0.0/16). This IP address is not bound to any physical network interface or container interface; it is purely a logical address maintained by the cluster's internal routing and forwarding mechanisms. The service associates a set of Pods via label selectors, and the IPs and ports of those Pods are collected into Endpoints. When clients access the service's ClusterIP, traffic is transparently distributed across these backends.

For NodePort-type services, external traffic first arrives at the node's physical NIC, then undergoes NAT translation by kube-proxy into Pod IPs before entering Calico's VXLAN forwarding plane. Calico itself does not implement service load-balancing logic but provides the foundational network layer for backend Pod reachability.

Calico enforces Pod-level access control on the host kernel using NetworkPolicy. It translates Kubernetes NetworkPolicy resources into iptables rules applied to the host's cali interfaces and forwarding paths. For inbound traffic to a Pod, rules can match source IP, destination port, protocol type, and label selectors to permit or deny traffic. Similarly, egress policies can be enforced on the host for outbound traffic. Because all Pod traffic must traverse the host's kernel forwarding plane, iptables rules can intercept packets either before they enter or after they exit the veth device. Calico leverages the Linux kernel's connection tracking (conntrack) mechanism to maintain connection state, allowing return traffic from established connections to pass automatically—reducing redundant rule matching overhead. On the VXLAN tunnel path, network policy enforcement occurs post-decapsulation: after a remote node's VXLAN packet is decapsulated, the inner packet enters the host's forwarding plane, where the kernel first evaluates iptables rules; only packets passing policy checks are forwarded to the target Pod's veth interface. Thus, security policies are enforced in the kernel of the destination node—not at the tunnel endpoint or source node.

External networks establish connectivity between the containerized application cluster and external networks, enabling external clients to access containerized applications. The Container Service provides two types of external networking capabilities: Service External Network and Pod External Network—designed respectively for load-balanced access and direct Pod access.

• Provides Service External Network functionality: assigns a dedicated Layer 3 network IP address to LoadBalancer-type services and associates it with a set of backend Pods, enabling external clients to access those backends via this IP through load balancing.
• Provides Pod External Network functionality: assigns a dedicated Layer 3 network IP address to each Pod, enabling external clients to directly access individual Pods using those IPs.

NodePort-type services expose ports to external clients but suffer from node IP non-HA and port-number limitations—making them unsuitable as the primary production solution. Service External Network integrates the MetalLB component and IP address management capabilities to resolve IP address HA and port exhaustion issues. When a user creates a LoadBalancer service, MetalLB automatically allocates an external VIP from a preconfigured address pool and writes it into the service status, enabling external clients to access backend Pods via that VIP. It supports two traffic ingress modes: Layer 2 mode uses ARP/NDP to bind the VIP to a single node for failover; BGP mode advertises the VIP to upstream physical routers via BGP, enabling true multi-node load distribution.

In Kubernetes, Pods receive only one cluster network interface and a single IP address by default—assigned by the primary CNI (Calico in this platform). However, in Underlay network scenarios, business requirements often demand that Pods be directly exposed on the physical network—or require multiple network interfaces to carry management, application, and storage traffic separately. Through coordinated use of Multus CNI, Macvlan, and Spiderpool, Pods gain the ability to acquire additional IP addresses.

Multus: Enables chaining multiple CNI plugins during Pod creation to attach extra network interfaces beyond the primary one.

Macvlan: Creates virtual sub-interfaces on the host's physical NIC and moves them directly into the Pod's network namespace, giving the Pod a Layer 2 interface bridged to the physical network—enabling direct communication with external networks without requiring a virtual bridge.

Spiderpool: Manages IP address allocation and deallocation for CNI plugins like Macvlan, providing address pool management, static IP assignment, node selection, and garbage collection—ensuring correct allocation and lifecycle consistency of additional IPs within the physical network.

This capability introduces additional operational complexity. Each additional IP consumes real address space from the physical network and must integrate with existing IP management systems (e.g., DHCP, CMDB, firewall policies). Network administrators must ensure the IP subnet is correctly configured on physical switches—including VLANs, gateways, and routes—otherwise, although the Pod interface receives an IP, it cannot communicate externally.

Container storage virtualization provides persistent data capabilities for containerized workloads. Based on the Kubernetes storage model, the Container Service abstracts host paths and integrated storage systems as data volumes mountable by Pods, enabling application data to persist independently of the container lifecycle and supporting the deployment and operation of stateful applications.

For ephemeral, node-specific, or system-component scenarios, HostPath can be used to directly access local node directories; for production workloads and cross-node scheduling scenarios, centralized storage, distributed storage, or other storage systems can be integrated via the Container Storage Interface (CSI).

HostPath allows mounting a directory from the host filesystem into a container's filesystem—ideal for use cases requiring node-local file access, such as log collection, node configuration reading, and running system-level components. This is achieved through bind mounts within the Mount Namespace, mapping a specified host filesystem path into the container.

When processes inside a container access a HostPath mount point, filesystem calls directly enter the host kernel's filesystem layer—bypassing block device translation, virtual disk drivers, and additional I/O virtualization layers—resulting in a highly direct access path. For example, container log collection can leverage HostPath mounts to read log files from the node's local filesystem, such as /var/log/pods.

HostPath is tightly bound to a specific node, lacks capacity virtualization boundaries, and is unsuitable as general-purpose persistent storage for cross-node migration scenarios. When using HostPath, careful planning is required—considering workload type, directory permissions, and node capacity—to prevent container writes from impacting host stability or interfering with other containers.

Integrated storage provides container workloads with persistent storage independent of node-local disks. By integrating with external storage systems, container volumes can be dynamically created, mounted, unmounted, expanded, and reclaimed according to business needs—reducing the impact of container recreation, host failures, or workload migration on application data.

The Container Service integrates with storage via Kubernetes-standard storage interfaces; specific capabilities depend on the underlying storage system and its driver implementation. Through mechanisms such as the CSI (Container Storage Interface), StorageClass, PersistentVolume (PV), and PersistentVolumeClaim (PVC), external storage volumes are incorporated into the Kubernetes resource model and exposed to Pods as data volumes.

Application and Resource Management organizes Kubernetes resources, application packages, and DevOps workflows into visual, maintainable application delivery objects. While preserving Kubernetes' standard resource model, Container Service provides graphical resource management, application lifecycle management, and engineering-grade delivery capabilities—enabling users to directly manage workloads, services, routes, network policies, and other foundational resources, or to complete comprehensive release processes via applications and pipelines.

Application Management enables deploying and maintaining a group of Kubernetes resources—including workloads, services, and storage—as a unified entity. Container Service leverages Helm to manage application instances, allowing users to perform releases, upgrades, rollbacks, version management, and runtime status inspection at the application level.

Applications involve the following core concepts:

• Helm: A Kubernetes package manager for defining, installing, and managing applications composed of Kubernetes resources. Container Service integrates Helm 3.0 and extends it with visual operations capabilities, reducing the complexity of authoring and maintaining application resource definitions directly.
• Application Package: Describes the workloads, container images, dependencies, and resource definitions required to deploy an application. Users can deploy a complete application using an Application Package and centrally manage all related application resources throughout its runtime lifecycle.

Container Service supports publishing open-source software from the Application Marketplace or Application Packages from local repositories to clusters, deploying full application instances, and providing application lifecycle management, runtime status visualization, and basic operations capabilities.

Standard Resource Management targets scenarios where users need direct control over native Kubernetes resources. In addition to deploying applications via Application Packages, Container Service offers graphical management for mainstream Kubernetes resources—including namespaces, workloads, jobs, Pods, Services, Ingresses, Network Policies, ConfigMaps, and Secrets.

Through Standard Resource Management, users can view and edit resource configurations without directly using command-line tools or YAML files. It also enables direct access to underlying Kubernetes resources when troubleshooting application anomalies, diagnosing resource relationships, or performing fine-grained adjustments.

DevOps Engineering integrates containerized application build, deployment, testing, and release processes into a unified engineering workflow. Built on the Kubernetes runtime foundation, it consolidates capabilities such as pipelines, environments, testing, templates, and performance metrics—enabling users to manage the full lifecycle of containerized applications across multi-cluster and multi-tenant environments.

The DevOps engineering solution comprises the following functional modules:

Artifacts and Delivery manages key objects in the container application lifecycle—from build artifacts to deployment configurations—including container images, application packages, and YAML templates. Through its local registry, application marketplace, and template management capabilities, Container Service organizes image storage, application publishing, and resource configuration into a unified delivery pipeline.

The local repository provides storage for container images and application packages, supporting the OCI container image standard. Users can upload, download, and manage artifacts via online upload, UI-based upload, or CLI commands, and control access scopes for different users through permission management.

Image Management also provides container-package, a component that captures software installation, configuration files, and data changes inside a container as a new image and pushes it to the platform's built-in local repository, enabling users to build and reuse container images.

Application package management is based on Helm for application deployment. Users can deploy applications using standard Helm methods or configure deployments via YAML parameters. For applications with numerous or complex parameters, the platform offers form-based configuration, displays differences between the form content and the original YAML, and injects the form-specified configurations back into the YAML during deployment.

Form-based configuration lowers the barrier to directly editing YAML, helping reduce formatting errors, missing configurations, and inconsistent parameter entries—enabling more stable delivery of application packages across different clusters or environments.

Kubernetes resources are typically deployed, updated, and inspected via YAML files. Template Management provides YAML templates for common resources such as workloads, services, and configuration sets, and also supports users in creating and maintaining custom YAML templates aligned with their business requirements.

Through Template Management, users can reuse common resource configurations, reducing redundant YAML authoring; for scenarios where a full Helm chart is not yet required, users can rapidly instantiate needed resources using templates.

The container service supports application traffic management and runtime analysis through microservice governance and service topology capabilities.

Istio is integrated to deliver service governance capabilities, with core features including traffic management, application topology visualization, and distributed tracing. When deploying a containerized application cluster, users may optionally enable service governance—triggering deployment of Istio control plane components onto the cluster's management node.

At Pod startup, the microservice governance component injects a sidecar container. Using iptables rules, this sidecar transparently redirects all inbound and outbound traffic from the application containers within the Pod to the service governance proxy. As a result, application containers operate under the illusion of direct external communication, while all traffic is actually routed first through the governance proxy, then distributed to specific backend instances according to predefined load-balancing policies—such as round-robin, least connections, or consistent hashing. Concurrently, telemetry data—including service request rate, success rate, and latency—is collected. The solution also supports canary (gray) releases for workloads.

After a Pod is managed by the microservice governance feature, telemetry data is collected and processed to visualize the real-time network topology among Pods and display basic monitoring metrics, providing an intuitive view of service invocation status.

The container service provides operational capabilities including health inspection, monitoring and alerting, log management, and event management.

The one-click health inspection feature performs comprehensive health checks on the platform, cluster infrastructure components, and key metrics and services for compute and network resources. It calculates a health score based on inspection results and offers additional capabilities such as automated inspection, login-free inspection, inspection recommendations, and inspection report export—designed to lower operational complexity and significantly improve O&M efficiency.

Users can customize health checks by category and perform one-click inspections. Upon initiation, the container service performs health checks on the selected resources or services. The one-click inspection feature includes a built-in health scoring mechanism that quantifies the health status of inspected resources or services, enabling users to intuitively and accurately assess the overall platform operational status.

The monitoring module comprises multiple components to collect and store metrics at the cluster, storage, workload, and container levels. In this design, Exporters periodically collect monitoring data; Prometheus aggregates the data; Thanos handles query and aggregation; and open-local provides persistent storage for monitoring data. The monitoring module consolidates key operational data—including cluster component status, application health statistics, and resource utilization—into unified, multi-metric dashboards for intuitive visualization of diverse resources and metrics, enabling rapid cluster status assessment. Additionally, it is compatible with the standard Prometheus API, supporting integration with mainstream monitoring systems such as Grafana.

The alerting module leverages metrics collected by the monitoring module. Users can flexibly define alert rules via a visual interface. After unified analysis of the metrics, alerts are delivered in real time through Alertmanager via browser notifications, email, WeCom (Enterprise WeChat), DingTalk, or Webhook—ensuring immediate notification to operations personnel. This significantly improves operational efficiency, reduces operational costs, and meets the requirements of most operational scenarios.

Kubernetes stores only container stdout/stderr logs locally on the node. Such local logs are tightly coupled to the node's lifecycle: historical logs are permanently lost upon node failure, disk corruption, or Pod eviction and recreation. The logging module consists of a log collector, log storage, and log viewer. It adopts a lightweight design and uses labels—not full-text search—as indexes, enabling faster, more efficient log indexing and substantially reducing index storage overhead. Key features include: unified collection and centralized display of container logs—including but not limited to stdout/stderr; and support for log classification and querying by namespace, workload, Pod, or container.

Kubernetes stores events in etcd and retains them by default for only one hour; expired events are automatically deleted, offering limited data retention. Event querying is a critical troubleshooting method for identifying anomalies in workloads, Pods, nodes, and storage mounts. The platform provides an event collection component that aggregates all Kubernetes cluster events and forwards them to the logging module, enabling persistent event storage and displaying details such as resource type, resource name, message, event type, and reason.