Scenario

• Product: ZCF
• Components: ZStack Cloud, ZStone
• ZStack Cloud version: 4.8.x
• ZStone version: 5.2.x
• Deployment: 3 hyperconverged nodes
• Trigger: Physical memory must be expanded one hyperconverged node at a time.

Symptoms

• Three hyperconverged nodes require hardware memory expansion.
• Workload and storage risks must remain controlled during the change.
• Each server must be powered off, upgraded, and verified independently.

Assessment

1. Verify ZStack Cloud and Ceph health before the change.
2. After the node starts, verify recognized memory with free -h.
3. After data recovery is restored and maintenance mode is exited, verify hosts and VMs are normal.

Cause

This is a hardware expansion scenario. Maintain one node at a time, stop its VMs, and stop storage recovery to reduce workload and data risks.

Resolution

1. Preparation

• Risk level: Medium.
• Plan a maintenance window and process nodes one at a time.
• Baseline configuration: 8 × 32GB per server.
• Target configuration: Server A 6 × 64GB; Server B 12 × 32GB; Server C 12 × 32GB.

2. Perform a health check.

• Storage UI: http://192.168.x:4000/
• ZStack Cloud UI: http://192.168.x:5000/
• Run on each host:

     ceph -s

• Expected result: Platform resources are connected and the Ceph cluster is HEALTH_OK.

3. Disable global high availability.

• Expected result: No unexpected high-availability action occurs during the change.

4. Shut down or migrate VMs on the target server and put the server into maintenance mode. Check:

   virsh list

• Expected result: No VM or virtual router is running on the target node.

5. Disable Data Recovery in the storage platform.

• Expected result: Automatic recovery and rebalancing do not interfere with the change.

6. Log in to the target server, power it off safely, and expand memory:

   sync && sync && poweroff

• Expected result: The server is fully powered off for the hardware change.

7. Start the server and verify out-of-band and operating-system memory recognition:

   free -h

• Expected result: The recognized memory matches the expected capacity.

8. Re-enable Data Recovery and wait for the cluster to become healthy.
9. Enable the host in ZStack Cloud and exit maintenance mode.
10. Start VMs on the node and verify normal operation.
11. Re-enable global high availability.
12. Repeat the same procedure for Servers B and C.

Verification

• Verify that ceph -s is healthy.
• Verify normal memory recognition with free -h.
• Verify that the host reconnects and exits maintenance mode.
• Verify normal VM operation.

Risks and rollback

• Risk: Incorrect memory recognition or slot configuration can prevent the node from starting.
• Risk level: Medium.
• If the node cannot start or cannot recognize memory after expansion, restore the original memory configuration, then restore data recovery, enable the host, and enable high availability as described in steps 8 through 11.