Source and classification

• Source: Internal Jira/TIC.
• Classification: Confirmed product mutual-exclusion constraint; advance UI/API notification and a controlled release action are unimplemented improvements.
• Confirmation basis: When third-party CBT and platform backup run concurrently on the same virtual machine, logs show block-job mutual exclusion. The related product improvement requires early API/UI detection and a controlled release action.
• Internal tracking: TIC-6011 and ZSV-12711.

Issue metadata

• Issue IDs: TIC-6011 and ZSV-12711
• Component code: ZSphere
• Component/product: ZVF / ZSphere
• Status: Open; product improvement not delivered
• Priority/severity: To be confirmed
• Reported version: ZSphere 4.10.25
• First fixed/affected version: Affected version is ZSphere 4.10.25; first fixed version to be determined
• Definition date: To be confirmed
• Engineering confirmation date: To be confirmed
• Last updated: September 2, 2026

Problem description

CBT, platform VM/disk backup, and migration use the same VM/disk block-job and NBD resources and cannot run concurrently. The platform can reject its own registered scheduling tasks, but an external Veeam task is not necessarily registered as a ZSphere scheduling task. The API precheck might therefore miss external use and fail only when the data plane creates a job. No verified one-click UI release action exists. Confirm that external block-job/NBD resources have been released before continuing platform backup or migration.

Current code evidence: CbtBackupApiInterceptor checks scheduling tasks associated with the VM and its disks and rejects CBT enablement if it finds VM, root-disk, or data-disk backup. Exporting/canceling NBD export also requires the VM to be Stopped. An external Veeam task is not necessarily registered as a ZSphere scheduling task, so the platform might not see the external use even though both paths run concurrently.

Symptoms

• Backup logs show expect N block jobs, only found M or CBT/backup mutual exclusion.
• The same policy sometimes succeeds and sometimes fails; changing rate limits does not reliably eliminate the issue.
• Management-node logs show The vm ... has already created a backup job, cannot enable the cbt task at the same time or a similar mutual-exclusion message.
• ZSphere Task Center might show only platform backup or migration failure and cannot directly indicate whether an external CBT task still uses the same VM.

Impact

• Platform backup, CBT, or migration for the same virtual machine might fail.
• A failed task might leave unreleased NBD/block-job resources, causing the next task to fail.
• Lowering the backup rate alone does not remove mutual exclusion. The recurrence risk can be reduced only by avoiding task overlap for the same VM and confirming that the resources have been released.

Workaround

• Verified self-service method: Stagger Veeam CBT and ZSphere platform backup/migration. For the same VM, pause scheduling in the other system, wait for the current task to finish, confirm no job is running, and then start the other task type. Merely staggering start times while long tasks overlap is insufficient.

Log identification (read-only)

1. In Veeam, record the job/session ID, VM UUID, and start/end times. In ZSphere Task Center, record the platform backup or migration apiId for the same VM. Times must be precise to the minute to determine actual overlap.
2. On the management node, query current and archived logs by platform-task apiId:

   KB_API_UUID='<apiId-from-ZSphere-task-details>'
   cd /usr/local/zstack/apache-tomcat/logs || exit 1
   ls -lh management-server.log*
   zgrep -a -n -C 3 "$KB_API_UUID" management-server.log*
   zgrep -a -n "$KB_API_UUID" management-server.log* | grep -iE 'json POST|http response|cbt|backup|nbd|block job'

Record the target ZSphere host/backup server, URI, taskuuid/taskUuid, and complete mutual-exclusion error from json POST and http response. If there is no usable apiId, search by VM UUID and exact time and obtain taskuuid from the request line.

3. On the addressed ZSphere host, run:

   KB_TASK_UUID='<taskuuid-from-management-server-log>'
   cd /var/log/zstack || exit 1
   zgrep -a -n -C 5 "$KB_TASK_UUID" zstack-kvmagent.log*

On the addressed image storage (ImageStore) backup server, run:

   KB_TASK_UUID='<taskuuid-from-management-server-log>'
   cd /var/log/zstack/zstack-store || exit 1
   zgrep -a -n -C 5 "$KB_TASK_UUID" zstore*
   zgrep -a -n "$KB_TASK_UUID" zstore* | grep -iE 'expect.*block jobs|only found|cbt|backup|nbd|error|fail'

Run each command set on its own node. Do not assume both logs exist on one machine because management-node logs show both targets. ZSphere-host and ImageStore requests might use different taskuuid values; use the value from each request line.

4. On the VM's ZSphere host, set the VM UUID and check for a current block job:

   KB_VM_UUID='<vm-uuid>'
   virsh qemu-monitor-command "$KB_VM_UUID" '{"execute":"query-block-jobs","arguments":{}}' --pretty

This issue applies if a Veeam CBT session and platform task overlap and logs show expect N block jobs, only found M or explicit CBT/backup mutual exclusion. Empty QMP after task completion only means no current job; retain logs from the failure window. To inspect current logs, use vim -R LOG_FILE, search with /, use n, and quit with :q!.

• Expected result: Only CBT or platform backup/migration exists in a time window. After the first task ends and releases NBD/block-job resources, the next task is created normally.
• Support-assisted method: If a failed task still holds resources, do not kill qemu-nbd or modify the database. Save both task IDs and QMP job/NBD state; ZStack Support releases/cancels them for the exact version. No verified one-click UI release button exists.
• Stop condition: Do not start the next task or perform manual cleanup if ownership cannot be uniquely determined, QMP still has an active job, NBD is still used, or the backup timelines cannot be aligned.

Affected products

• Product line: ZVF
• Product/component: ZSphere
• Affected version: Reported on ZSphere 4.10.25; confirm other versions from logs for the same scenario.
• Environment: The same VM is used concurrently by Veeam/third-party CBT and ZSphere platform backup or migration.
• Unaffected scenario: CBT, platform backup, and migration for the same VM are strictly staggered, and block-job/NBD resources are released after the preceding task.

Fix status

• Current status: Open; product improvement not delivered.
• Fixed version: To be determined.
• Related patch/fix package: No specific patch or fixed version.
• Related defect/fix record: ZSV-12711 tracks advance API/UI conflict detection, a release action, and acceptance criteria.

Related information

• Related TIC/Jira: TIC-6011 and ZSV-12711.
• The related product record only proposes a UI button, conflict scope, and acceptance criteria; it must not be described as an existing fix.