Environment

• Product: ZCF
• Component: ZCE-X
• Objects: Object storage data pool, Ceph OSD rebalance, and recovery QoS
• Version: 6.x
• Trigger condition: An object storage data pool must be rebalanced while recovery QoS remains at low speed to reduce the impact on workloads.

Requirement background

• The object storage data pool must be rebalanced.
• To reduce the impact of recovery traffic on workloads, set the object storage data pool recovery QoS to low speed in advance.

Confirmation method

1. Confirm that the current environment is an object storage scenario running ZCE-X 6.x.
2. Check whether the current recovery QoS of the object storage data pool is set to low speed.
3. Confirm that data recovery is disabled before running the rebalance command.

Approach

The goal of this operation is to rebalance the object storage data pool. If recovery QoS is not limited to low speed in advance, the recovery process may consume bandwidth available to workloads. Therefore, disable data recovery first and limit the recovery speed.

Procedure

1. Prerequisites

• Risk level: Low
• Before the change, confirm that the object storage data pool recovery QoS is set to low speed.
• Perform the operation during a coordinated maintenance window to prevent recovery traffic from affecting workloads.

2. Procedure

1. Log in to the storage platform, disable data recovery, and reconfirm that the object storage data pool recovery QoS is set to low speed.

• Expected result: Data recovery is disabled, and recovery QoS is set to low speed.

2. Run the object storage data pool rebalance command.

   ceph osd reweight-by-remap 101 1024 pool-5abace84a5174757ae84a2124429d9c7

• Expected result: The rebalance task starts.

3. Log in to the storage platform and enable data recovery again.

• Expected result: Data recovery is enabled again, and the cluster enters the recovery and balancing state.

Verification

• Confirm that the object storage data pool recovery QoS remains at low speed.
• Monitor the Ceph rebalance progress and its impact on services.
• Confirm that workloads are operating normally without significant abnormal latency.

Risks and rollback

• Risk: Recovery traffic consumes bandwidth available to workloads. If QoS is not limited to low speed, the impact on workloads may increase.
• Risk: Running the rebalance operation during an unsuitable window may affect service stability.
• Rollback: No separate rollback command is provided. If an exception occurs, pause recovery operations and contact technical support for assistance.