Centralized Storage
Centralized storage stores data on a central system consisting of one or more servers. Services are deployed centrally, and data is managed through the storage controllers. Centralized storage includes DAS, NAS, and SAN. You can select a storage type according to workload and data storage requirements.
- High Performance and High Reliability: Centralized storage keeps data in a centrally managed system, reducing the risks associated with fragmented storage and improving storage performance and reliability.
- Scalability: Centralized storage can expand over the network as data volume and service requirements grow.
- Security: Access control, data encryption, and other security measures protect data confidentiality and integrity.
The storage controller is the core component of a centralized storage architecture. A storage system typically has two controllers that provide redundancy, preventing a single controller failure from making the entire system unavailable. Frontend ports provide storage services to servers, while backend ports connect additional storage devices to expand system capacity and form a larger storage pool.
SAN
ZSphere supports various centralized storage systems. With SAN storage, LUNs allocated on the storage system can be used directly as storage pools for workload VMs. Compared with file system-based data storage, SAN storage is straightforward to deploy, scales flexibly, and can deliver the performance of the underlying physical disks. Supported shared-access protocols include iSCSI, FC, and NVMe-oF.
- Block-Level Access: SAN divides data into blocks, each with a unique identifier. Servers read from or write to specific data blocks using these identifiers.
- Parallel Access: Multiple nodes can simultaneously access different blocks within SAN, enabling parallel read and write operations.
- Shared Access: Multiple nodes can share the same SAN, allowing concurrent access to the same data blocks. This is crucial for scenarios requiring data sharing in distributed systems.
- Fault Tolerance: SAN typically provides data redundancy and failure recovery mechanisms to ensure data safety and reliability.
- Scalability: SAN can scale capacity and performance by adding more storage nodes.
- Performance Optimization: SAN usually uses various techniques, such as caching and load balancing, to optimize performance.
In this architecture, a volume on the centralized storage system is mapped to the hosts. Depending on how the storage controllers are connected, the multipath service on each host automatically aggregates SCSI devices with the same WWID into a multipath device. ZSphere detects multipath devices with the same WWID across all hosts in the cluster and creates a shared volume group (VG) for the selected volume. ZSphere then creates logical volumes (LVs) from the VG. These LVs correspond to volumes, snapshots, and other resources in ZSphere. To maintain storage cluster consistency, ZSphere provides the Sanlock shared-storage locking mechanism for storage heartbeat maintenance, node management, and metadata management.
Sanlock + Lvmlockd
Sanlock, a shared-storage lock manager, implements a lease mechanism based on Lamport algorithms such as Delta Paxos and Disk Paxos. The mechanism provides cluster membership management, heartbeat maintenance, arbitration, and message exchange. Lvmlockd converts these leases into locks that LVM can use.
With Sanlock and Lvmlockd, most LVM commands issued by a server are sent to Lvmlockd first. Lvmlockd translates the LVM operations into lock operations that Sanlock can process.
Different resource operations require different lock permissions. Resource operations proceed only after obtaining the appropriate locks, ensuring the security of metadata and data. This entire process requires no intermediate node coordination or centralized operations, forming a fully distributed architecture.
