Data Protection

Snapshot Management

ZStack ZSphere supports both Redirect-On-Write (ROW) and Copy-On-Write (COW) snapshot mechanisms.

For centralized storage: Local storage, NFS, and SAN storage use QCOW2 external snapshot, which is a type of ROW snapshot mechanism.
For distributed storage: Ceph enterprise edition use ROW snapshots. Self-Developed Distributed Storage uses COW snapshots.

Centralized Storage Snapshot Mechanism

This section introduces QCOW2 external snapshots.

Snapshot Chain and Snapshot Tree
Typically, a single disk corresponds to one snapshot chain. ZStack ZSphere supports creating a snapshot tree for a disk, where each branch of the tree represents a distinct snapshot chain.

图 1. Snapshot Tree
A snapshot tree includes the following information:
- Snapshot Chain: A relational chain composed of a set of snapshots for a disk. Each branch of the snapshot tree is a snapshot chain.
- Snapshot Node: An individual node in a snapshot chain, representing a single snapshot of the disk.
- Snapshot Size: The storage space consumed by a snapshot. You can view the total size of all snapshots in the snapshot tree and the size of an individual snapshot node.
Note:
- For non-distributed storage, the system allows a maximum of 128 nodes per snapshot chain by default. For distributed storage, the maximum number of snapshots per disk is 32, including both manual and automatic snapshots.
- When a snapshot chain reaches its maximum length:
  - If you continue to create automatic snapshots, the system automatically deletes the earliest automatic snapshot.
  - If you continue to create manual snapshots, you must manually delete unneeded snapshots.
- In production, we recommend that you keep the number of snapshots for an individual virtual machine below 5. Excessive snapshots can affect the VM performance, data security, and data storage capacity.
Create Snapshots
When an external snapshot is created, a new empty QCOW2 file is generated. This new file has its backing file pointed to the original QCOW2 file. The original QCOW2 file is set to read-only, effectively turning it into a snapshot itself. Subsequent data writes are directed only to the new QCOW2 file. The creation of an external snapshot involves creating a new blank qcow2 file.
- Create a single snapshot chain based on a backing file.
  
  图 2. Create Snapshot-Single Chain
  Assume there is an original base image (Base). A virtual machine 1 is created using this base image as a template. Then, snapshot 1A and 1B are created sequentially for virtual machine 1.
  - Base Image: A pre-made disk image file containing a complete operating system and bootloader, serving as the Base (read-only).
  - VM 1: A new empty file, Overlay-1, is created. Its backing file points to Base. Base remains read-only, thus becoming a snapshot. Subsequent data is written only to Overlay-1.
  - Snapshot 1A: A new empty file, Overlay-1A, is created. Its backing file points to Overlay-1. Overlay-1 is set to read-only, thus becoming a snapshot. Subsequent data is written only to Overlay-1A.
  - Snapshot 1B: A new empty file, Overlay-1B, is created. Its backing file points to Overlay-1A. Overlay-1A is set to read-only, thus becoming a snapshot. Subsequent data is written only to Overlay-1B. virtual machine 1 uses the disk file corresponding to the last snapshot 1B in the snapshot chain, and snapshot 1B is Active.
- Create multiple snapshot chains based on backing files.
  
  图 3. Create Snapshot-Multiple Chains
  Assume there is an original base image (Base). virtual machine 1, virtual machine 2, and virtual machine 3 are created using this base image as a template. Then, snapshot 1A and snapshot 1B are created sequentially for VM 1, snapshot 2A is created for VM 2, and snapshot 3A is created for VM 3.
  - Base Image: A pre-made disk image file containing a complete operating system and bootloader, serving as the Base (read-only).
  - Snapshot Chain 1:
    - Virtual Machine 1: A new empty file, Overlay-1, is created. Its backing file points to Base. Base remains read-only, thus becoming a snapshot. Subsequent data is written only to Overlay-1.
    - Snapshot 1A: A new empty file, Overlay-1A, is created. Its backing file points to Overlay-1. Overlay-1 is set to read-only, thus becoming a snapshot. Subsequent data is written only to Overlay-1A.
    - Snapshot 1B: A new empty file, Overlay-1B, is created. Its backing file points to Overlay-1A. Overlay-1A is set to read-only, thus becoming a snapshot. Subsequent data is written only to Overlay-1B. VM 1 uses the disk file corresponding to the last snapshot 1B in snapshot chain 1, and snapshot 1B is Active.
  - Snapshot Chain 2:
    - Virtual Machine 2: A new empty file, Overlay-2, is created. Its backing file points to Base. Base remains read-only. Subsequent data is written only to Overlay-2.
    - Snapshot 2A: A new empty file, Overlay-2A, is created. Its backing file points to Overlay-2. Overlay-2 is set to read-only, thus becoming a snapshot. Subsequent data is written only to Overlay-2A. VM 2 uses the disk file corresponding to the last snapshot 2A in snapshot chain 2, and snapshot 2A is Active.
  - Snapshot Chain 3:
    - Virtual Machine 3: A new empty file, Overlay-3, is created. Its backing file points to Base. Base remains read-only. Subsequent data is written only to Overlay-3.
    - Snapshot 3A: A new empty file, Overlay-3A, is created. Its backing file points to Overlay-3. Overlay-3 is set to read-only, thus becoming a snapshot. Subsequent data is written only to Overlay-3A. VM 3 uses the disk file corresponding to the last snapshot 3A in snapshot chain 3, and snapshot 3A is Active.
Merge Snapshots
External snapshots are interdependent, where each overlay file depends on its backing file. Each snapshot preserves its corresponding data, preventing the direct deletion of an individual snapshot to shorten the chain length. The chain length of external snapshots can be reduced through two methods: downward merging (Blockcommit) or upward merging (Blockpull).
- Blockcommit
  Within the same snapshot chain, you can merge overlays to backing files.
  
  图 4. Blockcommit
  
  Assume there is an original base image (Base). Virtual machine 1 is created based on Base, and three interdependent external snapshots are created for virtual machine 1: Snapshot 1A, Snapshot 1B, and Snapshot 1C. Now, Snapshot 1A and Snapshot 1B are merged downward into virtual machine 1. As a result, the backing file of Snapshot 1C (Active) points directly to virtual machine 1, so the snapshot chain is shortened. Snapshots 1A and 1B are no longer useful and can be deleted.
- Blockpull
  Within the same snapshot chain, you can merge backing files to overlays.
  
  图 5. Blockpull
  
  Assume there is an original base image (Base). Virtual machine 1 is created based on Base, and three interdependent external snapshots are created for virtual machine 1: Snapshot 1A, Snapshot 1B, and Snapshot 1C. Now, Snapshot 1A and Snapshot 1B are merged upward into Snapshot 1C (Active). As a result, the backing file of Snapshot 1C (Active) points directly to virtual machine 1, so the snapshot chain is shortened. Snapshots 1A and 1B are no longer useful and can be deleted.

Distributed Storage Snapshot Mechanism

Ceph enterprise edition uses ROW snapshot technology. Self-Developed Distributed Storage uses COW snapshots. For more information, see Volume Snapshot Protection.

Backup Management

Data Backup

The backup service supports data backup based on the QEMU block device layer. virtual machines on all types of data storage support backup operations. Backup types include full backup and incremental backup. A full backup contains a complete data set, while an incremental backup contains only the data updated since the last backup. Both full and incremental backups only backup actual used data.

By default, the backup policy is configured to automatically perform a full backup after every 63 incremental backups, starting after the initial full backup. Due to the dependencies between incremental backups, a new full backup must be created before previous incremental backups can be deleted. In practice, the system employs more intelligent and flexible strategies to determine the appropriate backup method, ensuring the safety and reliability of the backup data.

The data backup process consists of three parts: data replication, data transfer, and data storage.

Data Replication

The Backup Service module utilizes the dirty data tracking functionality (Dirty Bitmap) at the QEMU block device layer to track and export backup data.

Locations in the virtual machine's disk where data changes occur are called dirty data locations. The Dirty Bitmap records all such locations in the virtual disk file that have been modified since the last backup. Based on these records, all data modified since the last backup (incremental backup data) can be exported.

Ultimately, the full backup file and subsequent incremental backup files form a complete backup chain to preserve the entire data set. Since the Dirty Bitmap resides in the memory of the QEMU process, the tracking information is lost after a VM reboot. Therefore, ZStack ZSphere automatically exports a full backup data after a VM reboot.

Data Transfer

Two different implementation schemes are supported for different virtualization component versions, differing primarily in how backup data is transferred.

Scheme 1: Data Over SSHFS. This scheme uses SSHFS to mount the backup directory of the remote backup storage on the compute node and imports the backup data into the backup storage. SSHFS is a simple FUSE Over SSH solution. The data channel is encrypted by the SSH session, and each backup job uses a dedicated SSHFS link.

Scheme 2: Data Over NBD. This scheme uses the NBD module on the backup storage to export a backup disk. Then, the compute node utilizes a QEMU block device job (Block-job) to write backup data directly to this disk.

Data Storage

The backup storage supports various storage media types, including SAN, NAS, disk arrays, and tape libraries.

The backup storage stores data in a chunked and deduplicated format. The backup data is split into 64MB chunks. A hash is calculated for each chunk, and an index is created. Data chunks with identical has values are stored only once.

Data Recovery

When restoring a virtual machine or volume from a local backup, the chunked data on the backup storage is merged and imported into the data storage. For non-distributed storage, the restored backup data is stored as a disk chain. For distributed data storage, the disk chain is merged into a single disk file for storage. For a new recovery, the disk data restored to the data storage serves as an image cache to create a new virtual machine or volume. For an overwrite recovery, the system updates the database record of the current virtual machine or volume with the new disk path on the data storage and then deletes the old virtual machine or volume files.

上一篇Resource Virtualization 下一篇O&M Management