Storage Resource

Overview

Storage Resource: A logical unit that provides storage services externally, consisting of a storage cluster.

Related Definitions

Storage Pool: A storage pool is a logical partition in a storage cluster, which consists of storage severs and data disks to store objects.
Data Disk: A data disk is a logical storage unit, with each data disk corresponding to one data process. Multiple data disks can form a storage pool based on replication or Erasure Coding (EC) mechanisms.

Storage Pool

Create a General Purpose Pool

On the main menu of ZStack ZStone, choose Storage Resource > Storage Pool. On the Storage Pool page, click Create Storage Pool.

You can create three types of storage pools:

Block Storage Pool
Object Storage Pool
File Storage Pool

Create a Block Storage Pool

Set the following parameters:

Name: Set the name for the storage pool.
Naming rules: 1-1288 characters long. A name can contain Chinese characters, letters, digits, spaces, hyphens (-), underscores (_), periods (.), parenthesis (), colons (:), and plus signs (+).
Type: Select Block Storage.
Role: The default role is Data Pool and does not support modification.
Data Security Policy：
- Type: The default type is Replicas and does not support modification.
- Replicas: Set the number of replicas for the storage pool in the 2-6 value range.
  Note: In production environments, we recommend setting at least 3 replicas to ensure data security.
- Level: Select the level of the failure domain (Server/Rack/Room) according to your topology plan.
- Data Disk: Select data disks based on the topology canvas.
  Note:
  - Selected data disks must meet the redundancy level requirements.
  - Select data disks of similar sizes if possible.

Create an Object Storage Pool

Set the following parameters:

Name: Set the name for the storage pool.
Naming rules: 1-128 characters long. A name can contain Chinese characters, letters, digits, spaces, hyphens (-), underscores (_), periods (.), parenthesis (), colons (:), and plus signs (+).
Type: Select Object Storage.
Role: Select the role of the storage pool (Data Pool, Index Pool, and Compound Pool).
Note:
1. Data Pool: Stores data.
2. Index Pool: Stores the index information of stored objects.
3. Compound Pool: Supports multi-purpose resuse and can be selected as an Index Pool or a Data Extra Pool in Storage Policy.
Data Security Policy：
- Type: Select data redundancy type (Replicas/EC).
  - If you select Replicas, set the following parameters:
    - Replicas: Set the number of replicas for the storage pool in the 2-6 value range.
    Note:
    - Storage pools with Index Pool or Compound Pool role only support one redundancy policy, that is, Replicas.
    - In production environments, we recommend setting at least 3 replicas to ensure data security.
  - If you select EC, set the following parameters:
    - EC Policy: Set the EC policy for storage (Recommended/Custom).
      - Recommended: Select from six recommended values: 2+1, 4+2, 8+3, 4+2:1, 8+2:1, 16+2:1.
      - Custom: Customize the EC policy. Enter the number of data and parity blocks.
        Note: Positive integers only. Make sure that the number of data blocks is greater than the number of parity blocks, and parity blocks do not exceed 4.
      Note:
      
      An EC policy consists of data blocks and parity blocks. Data blocks indicate the number of data shards, while parity blocks indicate the number of parity shards generated through the algorithm. Taking the 4+2 EC policy on the server level as an example. This policy ensures data availability even when 2 servers fail.
      
      Disk Utilization is displayed in real time. The formula for calculating disk utilization: data blocks/(data blocks + parity blocks).
- Level: Select the level of the failure domain (Server/Rack/Room) according to your topology plan.
- Data Disk: Select data disks to add based on the topology canvas.
  Note:
  - Selected data disks must meet the failure domain requirements of the data security policy.
  - Select data disks of similar sizes if possible.

Create a File Storage Pool

Set the following parameters:

Name: Set the name for the storage pool.
Naming rules: 1-1288 characters long. A name can contain Chinese characters, letters, digits, spaces, hyphens (-), underscores (_), periods (.), parenthesis (), colons (:), and plus signs (+).
Type: Select File Storage.
Role: Select the role of the storage pool (Data Pool/Metadata Pool).
Data Security Policy:
- Type: The Replicas type has been selected by default and you cannot modify it.
- Replicas: Set the number of replicas for the storage pool in the 2-6 value range.
  Note: In production environments, we recommend setting at least 3 replicas to ensure data security.
- Level: Select the level of the failure domain (Server/Rack/Room) according to your topology plan.
- Data Disk: Select data disks to add based on the topology canvas.
  Note:
  - Selected data disks must meet the data redundancy level requirements.
  - Select data disks of similar sizes if possible.
  - To create a Metadata Pool, you need to use a raw SSD data disk.

Manage a General Purpose Pool

On the main menu of ZStack ZStone, choose Storage Resource > Storage Pool. Then, the Storage Pool page is displayed.

The following actions help to manage storage pools:

Action	Description
Create Storage Pool	Create storage pools.
Add Data Disk	Add one or more data disks to the storage pool.
Remove Data Disk	Remove a data disk to break the association with the storage pool. Hence a reduced storage pool capacity. Removing data disks may cause data losses. Proceed with caution. Note that you cannot remove data disks if: The storage pool is in a creating, deleting, or initializing status. The storage pool has a single replica. The data disk(s) to be removed has the only replica. When removing data disks in bulk, the rest of the data disks cannot meet the data security requirements. The cluster is enabled with Data Recovery and the pool capacity utilization may exceed the Backfill threshold after the removal. Note: If the pool capacity utilization is projected to exceed the Backfill threshold and you must remove the data disk(s), you can temporarily disable Data Recovery. The solution may cause data loss. Proceed with caution.
Set Recovery QoS	Choose the type of Recovery QoS for storage pools: Static QoS (Low Speed), Static QoS (Mid Speed), Static QoS (High Speed). When recovering the pool data, you can check data to recover, recovery rate, and remaining time on the General Purpose Pool page. Low-Speed Recovery gives a higher priority to the business bandwidth. The recovery time is relatively long. Any hardware failures during recovery may reduce the data security level. We recommend that you choose Low-Speed Recovery in a production environment. Mid-Speed Recovery gives the same priority to the business bandwidth and recovery bandwidth. The recovery time is medium. A saturated performance may increase the I/O latency. High-Speed Recovery gives a higher priority to the recovery bandwidth. The recovery time is relatively short. A saturated performance may affect business performance. Note: You can specify whether to select high-speed recovery QoS for a storage pool in Global Setting. Default value: false. If set to true, you can select high-speed recovery when setting recovery QoS for a storage pool. Mid-Speed Recovery and High-Speed Recovery may affect your business and we do not recommend that you use them in a production environment. Contact technical support for a risk evaluation in advance if you have to choose Mid-Speed Recovery or High-Speed Recovery.
Check Data Consistency	Set check policy to execute data consistency checks in the storage pool. We support two check policies: Default: The data consistency is checked once between 02:00 to 06:00 every day. Per Day: The data consistency is checked once in a custom time range per day. Note: A starting time equal to or greater than the ending time means the data consistency is checked between the starting time every day and the ending time on the next day. We recommend that you avoid busy business hours to execute data consistency checks. The check time accord with the server time. If the server time is not in sync with the browser time, data consistency checks will not be affected.
Rebalance Data Blocks	Manually rebalance the data blocks in the storage pool. Note: You must enable Data Recovery for clusters and ensure that the storage pool is in a healthy state before performing this operation. Rebalancing data blocks affects business performance. We recommend that you avoid busy business hours to perform this action.
Modify Data Security Policy	Modify the data security policy for storage pools. You are not provided with an option that does not meet the data security requirements. Replicas: For example, if the data redundancy level of the storage pool is server and the storage pool has 3 servers associated, you can set 2-3 replicas for the storage pool. You are not provided with an option that makes the pool capacity utilization exceed the Backfill threshold after the modification. If the pool capacity utilization has exceeded the Backfill threshold, you cannot set more replicas. Erasure Code (EC): You can only switch from a collapsed EC policy to a standard EC policy. Note: Modifying data security policy causes data migration which affects your business performance. We recommend that you avoid busy business hours to perform this action. Note that the Backfill threshold is 85% by default. You can modify this value in Global Setting. You can specify whether to set the number of replicas to 1 in Global Setting. Default value: false. If set to true, you can set the number of replicas to 1 when creating a storage pool or modifying replicas. If the storage pool operates in single-replica mode with insufficient capacity for replica modification, you cannot change the replica value. When setting more replicas, you must verify the required total PG quantity after modification does not exceed the sum of maximum PG capacities across all data disks.
Force Flush	On the details page of a storage pool, click Associated Resource > Data Disk to enter the Data Disk list. You can force flush to rapidly write dirty cache data to backend storage. Note: You can only force flush healthy hybrid-type data disks. During the force flush operation, the performance of the storage pool will significantly degrade. Proceed with caution.
Delete	Delete the existing storage pools. Deleting a storage pool detaches all data disks from the pool. After deletion, the storage pool data cannot be recovered. Proceed with caution. Note: Delete the storage pools of Block Storage type: Before deletion, you have to ensure that: No block storage volumes exist in the storage pool. The cluster where the storage pool is located is in the healthy state. Delete the storage pool of Object Storage type: For the storage pool chosen for initializing object storage, you have to check if it is associated with any resource, such as Storage Policy, Object Gateway, Object User, and Bucket. If the answer is no, you can delete it. Deleting the storage pool will cause the loss of the object storage system resources pool. After deletion, you cannot get the object storage service and access the data within. Proceed with caution. For the storage pool chosen for initializing object storage, you have to check if it is associated with any resource, such as Storage Policy, Object Gateway, Object User, and Bucket. If the answer is yes, you cannot delete it. For the storage pool not chosen for initializing object storage, you have to check if it is associated with any Storage Policy. If the answer is yes, you cannot delete the pool. Delete the storage pool of File Storage type: If a storage pool is associated with a file system, it cannot be deleted. If a storage pool is in a creating, scaling-in, scaling-out, updating, or deleting status, it cannot be deleted.

Create a High-Performance Pool

On the main menu of ZStack ZStone, choose Storage Resource > Storage Pool > High-Performance Pool. On the High-Performance Poolpage, click Create Storage Pool.

We support creating a high-performance storage pool of Block Storage type. Set the following parameters:

Name: Set the name for the storage pool.
Naming rules: 1-1288 characters long. A name can contain Chinese characters, letters, digits, hyphens (-), underscores (_), periods (.), parentheses (), colons (:), and plus signs (+).
Type: The default pool type is Block Storage.
Role: The default role is Data Pool.
Data Redundancy Type：
- Type: The default redundancy type is Replicas.
- Replicas: 3 replicas are supported.
- Data Disk: Select the data disk to add.
  Note: Ensure the selected data disks use the same interface protocol.

Manage a High-Performance Pool

On the main menu of ZStack ZStone, choose Storage Resource > Storage Pool > High-Performance Pool. Then, the High-Performance Pool page is displayed.

The following actions help to manage high-performance storage pools.

Action	Description
Create Storage Pool	Create high-performance storage pools.
Add Data Disk	Add one or more data disks to the storage pool. Note: Ensure that the added data disk share the same interface protocol with other data disks within the pool.
Remove Data Disk	Remove a data disk to break the association with the storage pool. Hence a reduced storage pool capacity. Note: Removing data disks causes data migrations that affect your business performance. We recommend that you avoid busy business hours to perform this action. Removing data disks may cause data losses. Proceed with caution. Note that you cannot remove data disks if: The storage pool is in a restoring, incorrect, and initializing state. After removal, the remaining data disks cannot meet the constraints for multiple replicas. When removing data disks in bulk, the rest of the data disks cannot meet the data security requirements. The cluster is enabled with Data Recovery and the pool capacity utilization may exceed the Backfill threshold after the removal.
Set Recovery QoS	Choose the type of Recovery QoS for storage pools: Static QoS (Low Speed), Static QoS (Mid Speed), Static QoS (High Speed). When recovering the pool data, you can check data to recover, recovery rate, and remaining time on the High-Performance Purpose Pool page. Low-Speed Recovery gives a higher priority to the business bandwidth. The recovery time is relatively long. Any hardware failures during recovery may reduce the data security level. We recommend that you choose Low-Speed Recovery in a production environment. Mid-Speed Recovery gives the same priority to the business bandwidth and recovery bandwidth. The recovery time is medium. A saturated performance may increase the I/O latency. High-Speed Recovery gives a higher priority to the recovery bandwidth. The recovery time is relatively short. A saturated performance may affect business performance. Note: You can specify whether to select high-speed recovery QoS for a storage pool in Global Setting. Default value: false. If set to true, you can select high-speed recovery when setting recovery QoS for a storage pool. Mid-Speed Recovery and High-Speed Recovery may affect your business and we do not recommend that you use them in a production environment. Contact technical support for a risk evaluation in advance if you have to choose Mid-Speed Recovery or High-Speed Recovery.
Delete	Delete the existing storage pools. Deleting a storage pool detaches all data disks from the pool. Note: To delete a high-performance pool requires meeting the following requirements: The storage pool is in a healthy, degrading, restoring, or incorrect state.。 No block storage volumes exist in the pool.

Object Storage System Resources Pool

On the main menu of ZStack ZStone, choose Storage Resource > Storage Pool > Object Storage System Resources Pool. Then, the Object Storage System Resources Pool page is displayed.

The object storage system resources pool is created to store metadata when initializing object storage and shares the same data disks and data security policy with the storage pool selected for initializing object storage.

Check Object Storage System Resources Pools

.rgw.root: Stores gateway configuration information, such as realm, zonegroup, and zone.
.rgw.control: Stores objects that are needed for internal notification mechanism, including notify.0-notify.7.
.rgw.log: Stores log information, such as gc, intent, usage, and reshard.
.rgw.meta: Stores metadata of the user, such as user.uid and user.keys.
.rgw.otp: Stores one-time password in multi-factor authentication.

Further Details About Storage Pool

On the main menu of ZStack ZStone, choose Storage Resource > Storage Pool. On the Storage Pool page, click one pool's name and enter its details page.

On the details page, four aspects of the storage pool are shown in card form: Basic Info, Capacity Statistics, Monitoring Data, and Capacity Monitoring.

Basic Info

You can check the basic information including state, type, role, pool UUID, the number of data disks, data redundancy type, and QoS type of the current storage pool.

Capacity Statistics

You can check Capacity Allocation and Capacity Usage of the current storage pool.

Capacity Allocation: Contains three parameters, namely, Raw Capacity, Data Redundancy Type, and Total Capacity.
Note:
- For pools of the Replicas policy type: Total Capacity = Raw Capacity ÷ Replica Number.
- For pools of the EC policy type: Total Capacity = Raw Capacity × Data Blocks ÷ (Data Blocks + Parity Blocks).
Capacity Usage: Contains four parameters, namely, Total Capacity, Capacity Used, Capacity Available, and Capacity Utilization.

Monitoring Data

You can check IOPS, bandwidth, and latency of the storage pool.
Time Span: 15 minutes, 1 hour, 6 hours, 1 day, 1 week, 1 year, and custom.

Capacity Monitoring

You can check both how much capacity has been used and how much capacity will be used in the next 30 days.

Note:

Only when a newly-created storage pool has been used for 12 hours can it be forecast the capacity.
This card statically displays data. You can obtain the latest data by refreshing the page or re-enter the same details page.

Data Disk

Create a Data Disk on General Purpose Node

On the main menu of ZStack ZStone, choose Storage Resource > Data Disk. On the Data Disk page, click Create Data Disk.

Set the following parameters:

Select Hard Disk: Select the hard disk to add as a data disk.
Note: You can add hard disks that are healthy, free, and of at least 25 GB size as data disks.
Cache Partition: Choose whether to set cache partition for data disks.
- Auto Mode: If you select auto mode, the system automatically attaches cache partitions provided by servers to the selected hard disks.
- Manual Mode: If you select manual mode, you can manually select cache partitions for each hard disk that has been selected.
Note: If available cache partitions are insufficient for the selected hard disks when you enable this parameter, part of the hard disks fail to be added as data disks.

Manage a Data Disk on General Purpose Node

On the main menu of ZStack ZStone, choose Storage Resource > Data Disk. Then, the Data Disk page is displayed.

The following actions help to manage data disks.

Action	Description
Create Data Disk	Create one or more data disks.
Set Maintenance Mode	Enable or disable the maintenance mode for data disks. A data disk in the maintenance mode is not involved in data block rebalances. Note: The maintenance mode stops services provided by and access to the data disk. To ensure system availability, we do not recommend that you put a data disk in the maintenance mode for a long time. A maintenance mode does not stop data migrations on the data disk. To stop data migrations, manually disable Data Recovery in Global Setting. Disable the maintenance mode and enable Data Recovery manually after you finish the disk maintenance. The services and data access automatically recover after you turn off the maintenance mode.
Delete	Deleting a data disk stops services provided by the data disks and deletes all relevant data. The deleted data cannot be recovered. Proceed with caution. Note: To delete a data disk requires meeting these requirements: The associated cache disk must be in a healthy state. The data disk must not be part of any storage pool.

Create a Data Disk on High-Performance Node

On the main menu of ZStack ZStone, choose Storage Resource > Data Disk. On the Data Disk page, choose High-Performance Node > Create Data Disk.

Note: To create a data disk on high-performance node, make sure you have planed a topology before. Otherwise, the creation will fail.

Set the following parameter:

Select Hard Disk: Select the hard disk to add as a data disk.
Note: Only Solid State Drives (SSDs) can be added as data disks.

Figure 1. Create a data disk on High-Performance Node

Manage a Data Disk on High-Performance Node

On the main menu of ZStack ZStone, choose Storage Resource > Data Disk > High-Performance Node. Then, the High-Performance Node page is displayed.

The following actions help to manage data disks.

Action	Description
Create Data Disk	Create one or more data disks.
Delete	Deleting a data disk stops services provided by the data disks and deletes all relevant data. The deleted data cannot be recovered. Proceed with caution. Note: To delete a data disk requires meeting one requirement: The data disk must not be part of any storage pool.

PreviousObject Storage NextHardware