文档中心API Explorer
ZHEN
文档中心›HCI›虚拟化版›用户手册HCI

Storage Service

You can rapidly deploy a distributed storage cluster or seamlessly take over an external distributed storage system. You can centrally manage and utilize distributed storage within the platform, maximizing existing storage resources. The platform provides storage monitoring capabilities, allowing direct access to distributed storage overviews, node details, performance metrics, and disk health status.

Deploy Distributed Storage in 3 Steps

Step 1: Upload Installation Package

Before you begin

  • You have prepared the installation package of the distributed storage software.
  • The platform management node must be added as a host.
  • At least 2 distributed storage nodes have been added as hosts to the platform, and the storage network has been configured for storage nodes using Kernel adapters from hosts or distributed port groups. For more information about configuring storage network, see Create a Kernel Adapter.
  • To ensure the distributed storage can achieve MN HA together with the platform, we recommend setting up management node HA in MN Ops before deploying the distributed storage. For more information about setting up management node HA, see Installation and Upgrade Tutorial.

Procedure

  1. In the navigation pane, choose Storage Service > Storage Overview.
  2. On the Storage Overview page, click Upload.
  3. In the Upload Installation Package dialog, set the following parameters:
    • Server IP: Display the IP address of the current management node.

      In a dual management node environment, this filed displays the IP address of the node where the VIP resides.

    • Storage Path: Specify a directory path on the server to store uploaded installation packages and their extracted files.
      Note:
      • Ensure the directory has sufficient available space. Otherwise, installation may fail.
      • Do not use system directories such as /, /dev, /proc, /sys, /usr/bin, /bin, or /opt. Using system directories may cause server instability.
    • Upload By: Select how to upload an installation package. Options include URL and Local File.
  4. Click OK.

Step 2: Deploy Management Service

Procedure

  1. On the Storage Overview page, click Install.
  2. In the Deploy Management Service dialog, set the following parameters:
    • VIP: Displays the management node IP address in a single management node environment, or the VIP address in a dual management node environment.
    • MN IP: Displays the IP address of single or dual management nodes. Enter the management node SSH port and password.
    • Database Password: If left blank, the initial database password will be used by default. If the password has been changed, enter the new password here.
  3. Review the configuration and click OK.

Step 3: Initialize Distributed Storage

Before you begin

Before proceeding, you need to set up the storage network for distributed storage nodes using Kernel adapters from hosts or distributed port groups. For more information about configuring storage network, see Create a Kernel Adapter.

Procedure

  1. On the Storage Overview page, click Initialize.
  2. In the Initialize Distributed Storage dialog, complete the initialization and storage configurations.
  3. For Initialization, set the following parameters:
    • Storage MN IP: Select the management node IP of the distributed storage.
    • Cluster: Display the cluster location.
    • Admin Network: Enter the admin network CIDR to manage and configure the storage cluster.
    • Cluster Network: Enter the cluster network CIDR to monitor data disks and synchronizing replicas among nodes in the storage cluster.
    • Public Network: Enter the public network CIDR for the storage cluster to provide external services.
    • Time Sync Server IP: This parameter is displayed based on the storage service type detected by the system. Enter the time synchronization server IP to ensure time synchronization between all nodes in the storage cluster.
    • Gateway Network: This parameter is displayed based on the storage service type detected by the system. Enter the gateway network to allow user services to access storage resources through this network.
  4. Review the configuration and click Next.
  5. For Storage Configuration, click Add to add distributed storage nodes.
  6. Configure manager and monitor roles.

    For data security, we recommend assigning the manager (Mgr) and monitor (Mon) role to 3 nodes.

  7. Specify a hostname for the distributed storage node.
    Note:
    • Length: 2 to 60 characters.
    • Allowed characters: Uppercase and lowercase letters, numbers, and hyphens (-).
    • No consecutive hyphens, nor start or end with a hyphen.
    • When adding nodes, a suffix (-1, -2, -3, and so on) is automatically appended to ensure uniqueness.
  8. Review the configuration and click OK.

What to do next

Starting from ZStack Cube Virtualization Edition 4.10.20, if you need to separately access the distributed storage management interface:
  • For ZCE-X v6.4.200.1, the default login credentials are admin/Admin@123
  • For ZCE-X v5, the default login credentials remain admin/password

Take Over Existing Distributed Storage

Take Over Distributed Storage

Once the storage package installation, network configuration, and system initialization are complete on the server, or if the distributed storage is already running and in use, you can choose this way to take over the distributed storage.

Procedure

  1. In the navigation pane, choose Storage Service > Storage Overview.
  2. On the Storage Overview page, click Take Over Existing Storage.
  3. In the Take Over Distributed Storage dialog, set the following parameters:
    • Storage MN IP: Enter the management node IP of the distributed storage to be taken over.
    • Storage Service Check: Click Check to check if storage services exist and whether the connection can be established successfully.
    • Username: Enter the username for logging into the distributed storage platform.
    • Password: Enter the login password.
    • Access Token: Fill in this parameter as needed based on the detected storage service type.
  4. Confirm the configuration and click OK.

What to do next

Starting from ZStack Cube Virtualization Edition 4.10.20, if you need to separately access the distributed storage management interface:
  • For ZCE-X v6.4.200.1, the default login credentials are admin/Admin@123
  • For ZCE-X v5, the default login credentials remain admin/password

Cancel Takeover of Distributed Storage

Canceling the takeover of distributed storage will not affect any stored data, but you will no longer be able to manage the distributed storage system.

Procedure

  1. In the navigation pane, choose Storage Service > Storage Overview.
  2. On the Storage Overview page, click Cancel Takeover.
  3. In the confirmation dialog, read the risk warning and click OK after you acknowledge the risk.

Distributed Storage Resource Management

Storage Pool

Create a General Purpose Pool

On the main menu of ZStack Cube Virtualization Edition, choose Storage Service > Storage Pool. On the Storage Pool page, click Create Storage Pool.

You can create three types of storage pools:
  • Block Storage Pool
  • Object Storage Pool
  • File Storage Pool

Create a Block Storage Pool

Set the following parameters:
  • Name: Set the name for the storage pool.

    Naming rules: 1-1288 characters long. A name can contain Chinese characters, letters, digits, spaces, hyphens (-), underscores (_), periods (.), parenthesis (), colons (:), and plus signs (+).

  • Type: Select Block Storage.
  • Role: The default role is Data Pool and does not support modification.
  • Data Security Policy：
    • Type: The default type is Replicas and does not support modification.
    • Replicas: Set the number of replicas for the storage pool in the 2-6 value range.
      Note: In production environments, we recommend setting at least 3 replicas to ensure data security.
    • Level: Select the level of the failure domain (Server/Rack/Room) according to your topology plan.
    • Data Disk: Select data disks based on the topology canvas.
      Note:
      • Selected data disks must meet the redundancy level requirements.
      • Select data disks of similar sizes if possible.
图 1. Create a Block Storage Pool


Create an Object Storage Pool

Set the following parameters:
  • Name: Set the name for the storage pool.

    Naming rules: 1-128 characters long. A name can contain Chinese characters, letters, digits, spaces, hyphens (-), underscores (_), periods (.), parenthesis (), colons (:), and plus signs (+).

  • Type: Select Object Storage.
  • Role: Select the role of the storage pool (Data Pool, Index Pool, and Compound Pool).
    Note:
    1. Data Pool: Stores data.
    2. Index Pool: Stores the index information of stored objects.
    3. Compound Pool: Supports multi-purpose resuse and can be selected as an Index Pool or a Data Extra Pool in Storage Policy.
  • Data Security Policy：
    • Type: Select data redundancy type (Replicas/EC).
      • If you select Replicas, set the following parameters:
        • Replicas: Set the number of replicas for the storage pool in the 2-6 value range.
        Note:
        • Storage pools with Index Pool or Compound Pool role only support one redundancy policy, that is, Replicas.
        • In production environments, we recommend setting at least 3 replicas to ensure data security.
      • If you select EC, set the following parameters:
        • EC Policy: Set the EC policy for storage (Recommended/Custom).
          • Recommended: Select from six recommended values: 2+1, 4+2, 8+3, 4+2:1, 8+2:1, 16+2:1.
          • Custom: Customize the EC policy. Enter the number of data and parity blocks.
            Note: Positive integers only. Make sure that the number of data blocks is greater than the number of parity blocks, and parity blocks do not exceed 4.
          Note:
          • An EC policy consists of data blocks and parity blocks. Data blocks indicate the number of data shards, while parity blocks indicate the number of parity shards generated through the algorithm. Taking the 4+2 EC policy on the server level as an example. This policy ensures data availability even when 2 servers fail.
          • Disk Utilization is displayed in real time. The formula for calculating disk utilization: data blocks/(data blocks + parity blocks).
    • Level: Select the level of the failure domain (Server/Rack/Room) according to your topology plan.
    • Data Disk: Select data disks to add based on the topology canvas.
      Note:
      • Selected data disks must meet the failure domain requirements of the data security policy.
      • Select data disks of similar sizes if possible.
图 2. Create an Object Storage Pool


Create a File Storage Pool

Set the following parameters:
  • Name: Set the name for the storage pool.

    Naming rules: 1-1288 characters long. A name can contain Chinese characters, letters, digits, spaces, hyphens (-), underscores (_), periods (.), parenthesis (), colons (:), and plus signs (+).

  • Type: Select File Storage.
  • Role: Select the role of the storage pool (Data Pool/Metadata Pool).
  • Data Security Policy:
    • Type: The Replicas type has been selected by default and you cannot modify it.
    • Replicas: Set the number of replicas for the storage pool in the 2-6 value range.
      Note: In production environments, we recommend setting at least 3 replicas to ensure data security.
    • Level: Select the level of the failure domain (Server/Rack/Room) according to your topology plan.
    • Data Disk: Select data disks to add based on the topology canvas.
      Note:
      • Selected data disks must meet the data redundancy level requirements.
      • Select data disks of similar sizes if possible.
      • To create a Metadata Pool, you need to use a raw SSD data disk.
图 3. Create a File Storage Pool


Manage a General Purpose Pool

On the main menu of ZStack Cube Virtualization Edition, choose Storage Service > Storage Pool. Then, the Storage Pool page is displayed.

The following actions help to manage storage pools:
Action Description
Create Storage Pool Create storage pools.
Add Data Disk Add one or more data disks to the storage pool.
Remove Data Disk Remove a data disk to break the association with the storage pool. Hence a reduced storage pool capacity.
Removing data disks may cause data losses. Proceed with caution. Note that you cannot remove data disks if:
    • The storage pool is in a creating, deleting, or initializing status.
    • The storage pool has a single replica.
    • The data disk(s) to be removed has the only replica.
    • When removing data disks in bulk, the rest of the data disks cannot meet the data security requirements.
    • The cluster is enabled with Data Recovery and the pool capacity utilization may exceed the Backfill threshold after the removal.
  • Note: If the pool capacity utilization is projected to exceed the Backfill threshold and you must remove the data disk(s), you can temporarily disable Data Recovery. The solution may cause data loss. Proceed with caution.
Set Recovery QoS Choose the type of Recovery QoS for storage pools: Static QoS (Low Speed), Static QoS (Mid Speed), Static QoS (High Speed). When recovering the pool data, you can check data to recover, recovery rate, and remaining time on the General Purpose Pool page.
  • Low-Speed Recovery gives a higher priority to the business bandwidth. The recovery time is relatively long. Any hardware failures during recovery may reduce the data security level. We recommend that you choose Low-Speed Recovery in a production environment.
  • Mid-Speed Recovery gives the same priority to the business bandwidth and recovery bandwidth. The recovery time is medium. A saturated performance may increase the I/O latency.
  • High-Speed Recovery gives a higher priority to the recovery bandwidth. The recovery time is relatively short. A saturated performance may affect business performance.
    Note: You can specify whether to select high-speed recovery QoS for a storage pool in Global Setting. Default value: false. If set to true, you can select high-speed recovery when setting recovery QoS for a storage pool.
  • Mid-Speed Recovery and High-Speed Recovery may affect your business and we do not recommend that you use them in a production environment. Contact technical support for a risk evaluation in advance if you have to choose Mid-Speed Recovery or High-Speed Recovery.
Check Data Consistency Set check policy to execute data consistency checks in the storage pool. We support two check policies:
  • Default: The data consistency is checked once between 02:00 to 06:00 every day.
  • Per Day: The data consistency is checked once in a custom time range per day.
Note:
  • A starting time equal to or greater than the ending time means the data consistency is checked between the starting time every day and the ending time on the next day.
  • We recommend that you avoid busy business hours to execute data consistency checks.
  • The check time accord with the server time. If the server time is not in sync with the browser time, data consistency checks will not be affected.
Rebalance Data Blocks Manually rebalance the data blocks in the storage pool.
Note:
  • You must enable Data Recovery for clusters and ensure that the storage pool is in a healthy state before performing this operation.
  • Rebalancing data blocks affects business performance. We recommend that you avoid busy business hours to perform this action.
Modify Data Security Policy Modify the data security policy for storage pools. You are not provided with an option that does not meet the data security requirements.
  • Replicas: For example, if the data redundancy level of the storage pool is server and the storage pool has 3 servers associated, you can set 2-3 replicas for the storage pool. You are not provided with an option that makes the pool capacity utilization exceed the Backfill threshold after the modification. If the pool capacity utilization has exceeded the Backfill threshold, you cannot set more replicas.
  • Erasure Code (EC): You can only switch from a collapsed EC policy to a standard EC policy.
Note:
  • Modifying data security policy causes data migration which affects your business performance. We recommend that you avoid busy business hours to perform this action.
  • Note that the Backfill threshold is 85% by default. You can modify this value in Global Setting.
  • You can specify whether to set the number of replicas to 1 in Global Setting. Default value: false. If set to true, you can set the number of replicas to 1 when creating a storage pool or modifying replicas.
  • If the storage pool operates in single-replica mode with insufficient capacity for replica modification, you cannot change the replica value.
  • When setting more replicas, you must verify the required total PG quantity after modification does not exceed the sum of maximum PG capacities across all data disks.
Delete Delete the existing storage pools. Deleting a storage pool detaches all data disks from the pool. After deletion, the storage pool data cannot be recovered. Proceed with caution.
Note:
Delete the storage pools of Block Storage type:
  • Before deletion, you have to ensure that:
    • No block storage volumes exist in the storage pool.
    • The cluster where the storage pool is located is in the healthy state.
Delete the storage pool of Object Storage type:
  • For the storage pool chosen for initializing object storage, you have to check if it is associated with any resource, such as Storage Policy, Object Gateway, Object User, and Bucket. If the answer is no, you can delete it. Deleting the storage pool will cause the loss of the object storage system resources pool. After deletion, you cannot get the object storage service and access the data within. Proceed with caution.
  • For the storage pool chosen for initializing object storage, you have to check if it is associated with any resource, such as Storage Policy, Object Gateway, Object User, and Bucket. If the answer is yes, you cannot delete it.
  • For the storage pool not chosen for initializing object storage, you have to check if it is associated with any Storage Policy. If the answer is yes, you cannot delete the pool.
Delete the storage pool of File Storage type:
  • If a storage pool is associated with a file system, it cannot be deleted.
  • If a storage pool is in a creating, scaling-in, scaling-out, updating, or deleting status, it cannot be deleted.

Storage Node

Add a General Purpose Storage Server

On the main menu of ZStack Cube Virtualization Edition, choose Storage Service > Storage Node. On the Server page, click Add Server.

Adding a general purpose storage server involves five steps. Set the following parameters to complete the server configurations.

Step One: Basic Configurations

  • Server IP Address: Enter the server IP address. You can specify either a single IP address or IP address range. We support adding multiple servers in bulk.
  • Type: Select server type. Two server types are supported:
    • Storage Server:
      • Provides storage pools with hard disks that can be used as data disks.
      • Supports five roles: Management, Monitor, Block Storage Gateway, Object Storage Gateway, and File Storage Gateway.
    • Storage Gateway Server:
      • Hosts various interfaces and clients. The system only manages the server gateways and does not manage hard disks on the server.
      • Supports only one role: Block Storage Gateway.
  • Role: Configure the role of servers. Five roles are supported:
    • Admin Role (Management):
      • Responsible for the collection and management of the runtime status of the cluster and manages the distributed storage cluster as the management node in multiple ways, such as GUI and API.
      • We recommend that you deploy at least 2 admin roles to meet the high availability requirement.
    • Monitor Role:
      • Responsible for monitoring the cluster storage data and maintaining overall status of the cluster, including metadata such as data mapping and cluster authentication.
      • We recommend that you deploy an odd number of monitor roles (3+2*N, N≥0) to meet the high availability requirement.
    • Block Storage Gateway:
      • Responsible for the access between the server and the storage cluster through Block interface.
      • By default, this role is selected for a storage server.
      • By default, this role is selected for a gateway server. And a storage gateway server only supports this role.
    • Object Storage Gateway:
      • Responsible for the access between the server and the storage cluster through Object interface.
      • To use object storage service, you need to select this role.
      • On an object storage gateway server, you can turn on the object gateway to provide the S3 protocol and gateway services.
    • File Storage Gateway:
      • To use file storage service, you need to select this role.
      • On a file storage gateway server, you can create a file gateway to provide file storage access protocols such as SMB and NFS.
    Note:
    • When you add a server for the first time, three roles including Management, Monitor, and Block Storage Gateway, are selected by default. The Block Storage Gateway role can be deselected, while Management and Monitor roles are required.
    • For subsequent server additions, you can add storage servers without roles.
    • Deploy at least three storage servers with Management, Monitor, and Block Storage Gateway roles in a cluster.
图 1. Basic configurations


Step Two: Environment Configurations

  • SSH Username: Enter the SSH username for the server. Default: root.
  • SSH Password: Enter the SSH password. The system uses this password only for password-free login configurations and does not store the password.
  • Port: Enter the server port number. Default port: 22.
  • Server Name: (Optional) Specify a server name.

    Naming rules: 1-63 characters long. The name can contain lower-case letters (a-z), digits (0-9), periods (.), and hyphens (-). Avoid starting with a hyphen or number as well as ending with a hyphen.

    Note:
    • If you do not set a server name, ensure the server name-to-IP mapping is preconfigured in /etc/hosts file. The system uses the existing server name after server addition.
    • If you set a new server name, it overwrites the existing name-to-IP mapping in /etc/hosts file.
    • When you add servers in bulk, the names of these servers will end with a suffix, that is, the last part of their IP address (0-254), to distinguish these servers, for example, server-24.
  • Time Sync Service: Choose to enable or disable the time synchronization service.

    If you enable this setting, the system synchronizes the newly-added server's clock with other servers in the cluster.

  • Password-Free Login: If you enable this setting, the system configures password-free logins to the server with the SSH username and password.
图 2. Environment configurations


Step Three: Network Configurations

  • Admin IP: Sets the IP address the management network which manages and configures storage clusters. The default admin IP is the server IP address.
  • Public IP: Sets the IP address of the public cluster network which facilitates interaction between block storage gateways and storage pools.
  • Cluster IP: Sets the IP address of the cluster internal network which monitors data disks across cluster servers and synchronizes replicas.
    Note: Skip setting Cluster IP when you add a storage gateway server.
图 3. Network configurations


Step Four: Confirm

Review the information of the server to add. You can navigate back to modify configuration details if needed.

图 4. Confirm


Step Five: Install

Check the server deployment progress. If you exit this page during installation, the process continues in the background. You can check the process via the operation log at any time.
Note:
  • You cannot exit the page when adding a server for the first time.
  • Avoid refreshing the browser when adding a server for the first time.
图 5. Install


Manage a General Purpose Storage Server

On the main menu of ZStack Cube Virtualization Edition, choose Storage Service > Storage Node > General Purpose Node. Then, the General Purpose Node page is displayed.

The following actions help to manage general purpose storage servers.
Action Description
Add Server Add one or more servers.
Note: If a monitoring server in the disconnected state exists in the cluster, you cannot add a new server.
Delete Server Deleting a server stops the services provided by the server and deletes all relevant data. Deleted data cannot be recovered. Proceed with caution.
Note: To delete a server requires meeting the following requirements:
  • The server is in the connected state.
  • The cluster where the server is located is in the healthy status.
  • No data disks are running in the server.
  • The server is not added as an Object Storage Gateway.
  • The server is not added as a Block Storage Gateway.

Data Disk

Create a Data Disk on General Purpose Node

On the main menu of ZStack Cube Virtualization Edition, choose Storage Service > Data Disk. On the Data Disk page, click Create Data Disk.

Set the following parameters:
  • Select Hard Disk: Select the hard disk to add as a data disk.
    Note: You can add hard disks that are healthy, free, and of at least 25 GB size as data disks.
  • Cache Partition: Choose whether to set cache partition for data disks.
    • Auto Mode: If you select auto mode, the system automatically attaches cache partitions provided by servers to the selected hard disks.
    • Manual Mode: If you select manual mode, you can manually select cache partitions for each hard disk that has been selected.
    Note: If available cache partitions are insufficient for the selected hard disks when you enable this parameter, part of the hard disks fail to be added as data disks.
图 1. Create a data disk


Manage a Data Disk on General Purpose Node

On the main menu of ZStack Cube Virtualization Edition, choose Storage Service > Data Disk. Then, the Data Disk page is displayed.

The following actions help to manage data disks.
Action Description
Create Data Disk Create one or more data disks.
Set Maintenance Mode Enable or disable the maintenance mode for data disks. A data disk in the maintenance mode is not involved in data block rebalances.
Note:
  • The maintenance mode stops services provided by and access to the data disk. To ensure system availability, we do not recommend that you put a data disk in the maintenance mode for a long time.
  • A maintenance mode does not stop data migrations on the data disk. To stop data migrations, manually disable Data Recovery in Global Setting.
  • Disable the maintenance mode and enable Data Recovery manually after you finish the disk maintenance. The services and data access automatically recover after you turn off the maintenance mode.
Delete Deleting a data disk stops services provided by the data disks and deletes all relevant data. The deleted data cannot be recovered. Proceed with caution.
Note: To delete a data disk requires meeting these requirements:
  • The associated cache disk must be in a healthy state.
  • The data disk must not be part of any storage pool.

Physical Hard Disk

Scan Hard Disks on General Purpose Node

On the main menu of ZStack Cube Virtualization Edition, choose Storage Service > Physical Hard Disk. On the Hard Disk page, click Scan and all hard disks on storage servers and their information will be displayed in the list.

图 1. Scan hard disks


Note:
  • Healthy free disks can be used for cache partitioning (SSD recommended) or added as data disks. If you set cache partition, ensure that each partition has a minimum capacity of 50 GB.
  • A scanned unknown disk implies that the disk contains unrecognized partitions.
    • For ZStack Cube Virtualization Edition 4.2.0 and earlier versions, run the wipefs -af /dev/sdX command to manually clean up partitions before rescanning the disk.
    • For ZStack Cube Virtualization Edition 4.2.0 and later versions, go to the Hard Disk page and click Initialize Hard Disk to clean up partitions directly.
  • In some hardware environments, newly-added hard disks may not be detected. Reboot the server or contact official technical support for assistance.

Manage Hard Disks on General Purpose Node

On the main menu of ZStack Cube Virtualization Edition, choose Storage Service > Physical Hard Disk. Then, the Hard Disk page is displayed.

The following actions help to manage hard disks.
Action Description
Scan Scan and list all hard disks in the server and their use.
Set Cache Partition You can set cache partitions for one or more healthy free disks.
Note:
  • Each partition has a minimum capacity of 50 GB.。
  • You can set up to 36 partitions.
  • If you fail to perform bulk action, select free disks of the same total capacity and try to set cache partition again.。
Clean up Cache We support cleaning up cache for cache disks in the healthy state.
Note:
  • Cleaning up cache means cleaning up all partitions in the cache disk. The clean-up cannot be recovered. Proceed with caution.
  • You can reset cache partition or add hard disks as data disks.
  • If the partitions of a cache disk is currently in use by data disks, you cannot clean up the cache disk. You need to delete the associated data disks first before trying to clean up cache again.
Initialize Hard Disk Initialize one or more disks in the healthy state whose use is unknown.
Note:
  • After initialization, the use of a hard disk will change from Unknown Disk to Free Disk.
  • Initializing hard disks will erase all existing partitions. This operation may cause permanent data loss. Proceed with caution.
Disk Light Enable or disable the disk light to quickly locate the hard disk.
Note:
  • You cannot light up a system disk, virtual disk, or offline disk.
  • We recommend you use the hard disk compatible with our platform, for example, HGST HUS728T8TALE6L4 or ST2000DM001-1ER164 and so on.
上一篇Bare Metal Management