Environment

• Product: ZCF
• Component: ZStone
• Objects: Distributed storage, storage management node, deep-scrub, and crontab
• Version: 5.2.6
• Trigger condition: Scheduled tasks are required to enable and disable deep-scrub for a storage pool during suitable deep-scrub windows.

Symptoms

• The deep-scrub function of a storage pool must be enabled and disabled on a schedule.
• The goal is to run deep-scrub during fixed time windows without affecting workloads.
• The change does not require restarting management services or hosts.

Diagnosis

1. Confirm that the current environment is within the scope of this article, and record key information such as the current version, target version, system architecture, operating system, deployment mode, and storage type.
2. Confirm that the target storage pool name can be obtained from the storage pool overview in the storage UI.
3. Confirm that the scheduled tasks are correctly added to crontab.
4. Check /root/ceph_deep_scrub_ctl.log and confirm that the script runs as expected.

Cause

The goal of this optimization is to use scheduled tasks on any storage management node to enable and disable deep-scrub, so that it runs during specified time periods and has less impact on workloads.

Resolution

1. Prerequisites

• Risk level: Low
• Disable global high availability before the change.
• Upload the control script to the /root/ directory of any storage management node.

2. Procedure

1. Disable global high availability.

• Expected result: The platform enters maintenance mode.

2. Upload the task script to any storage management node.

• Script name: ceph_deep_scrub_ctl.sh
• Expected result: The script is located in the /root/ directory.

3. Configure scheduled tasks to enable and disable deep-scrub.

   crontab -e
   0 22 * * * /bin/bash /root/ceph_deep_scrub_ctl.sh on $POOL_NAME
   0 6 * * * /bin/bash /root/ceph_deep_scrub_ctl.sh off $POOL_NAME

• Parameters:
• on: Enables deep_scrub.
• off: Disables deep_scrub.
• $POOL_NAME: Storage pool name obtained from the storage pool overview in the storage UI.
• Log location: /root/ceph_deep_scrub_ctl.log
• Expected result: The scheduled tasks are configured successfully.

4. After confirming that the cloud platform is operating normally, enable global high availability again.

• Expected result: High availability is restored, and workloads operate normally.

Verification

• Confirm that the scheduled tasks exist in crontab.
• Check /root/ceph_deep_scrub_ctl.log and confirm that deep-scrub is enabled and disabled according to schedule.
• Confirm that the platform and workloads operate normally after high availability is restored.

Risks and rollback

• Risk: The solution is assessed as low risk, and the change does not affect workloads.
• Risk level: Low.
• Rollback: Delete the configured crontab tasks and the /root/ceph_deep_scrub_ctl.sh and /root/ceph_deep_scrub_ctl.log files.

Exception handling recommendations

If an exception occurs during the change, stop subsequent operations immediately and restore the environment according to the rollback plan. If necessary, delete the scheduled tasks and script to restore the original state.