DocsKnowledge BaseAPI Explorer
Documentation›ZCF›ZStack Cloud›Explore more›CLI Command ManualZCF
Document navigation

GPU Device

Overview

A GPU device is a microprocessor with high parallel computing capabilities that can be used to accelerate large-scale computing workloads. In addition to graphics rendering tasks, GPUs have become core hardware for AI training and inference, effectively improving the efficiency of deep learning, large model inference, data processing, and other business operations. Through technologies such as GPU passthrough, vGPU, dGPU, whole-card scheduling, and VRAM slicing, GPUs can provide elastic computing resources for VM instances and containers in the form of entire cards or fine-grained slices, enhancing the throughput, response speed, and resource utilization of AI services.

GPU Technologies

ZStack Cloud supports multiple GPU allocation methods, allowing GPUs to be attached to VM instances or containers using the following technologies:
  • VM Instance
    • GPU Passthrough

      Passthrough a physical GPU along with its associated peripheral devices (such as graphics cards, sound cards, inference cards, etc.) as a unified unit directly to a VM instance. The VM instance exclusively occupies the physical GPU and gains access to its powerful parallel computing capabilities.

      The following GPU models support passthrough:
      Vendor Model
      NVIDIA Nvidia RTX 6000Ada, Nvidia RTX A6000
      GeForce RTX 5090, Geforce RTX 4090, Nvidia RTX 3090
      Quadro RTX 8000, Quadro RTX 6000
      M4000, P2000
      GTX 1650/1660, GTX 1060ti
      H100, H200, H800, H20
      Note: Supported only by ZStack Cloud H84R ISO.
      Nvidia L40, Nvidia L20, Nvidia L4
      Nvidia A100, Nvidia A30
      Nvidia A40, Nvidia A16, Nvidia A10, Nvidia T4
      Tesla V100, Tesla P4/6/40/100, M6/10/60
      K6000
      AMD Radeon v620, Radeon RX5700
      RRO W7800
      Note: Passthrough to VM instances running Windows operating system is not currently supported.
      FirePro S7150, FirePro S7150X2
      Huawei Atlas 300i pro
      Note: Only GPUs of this model installed on ARM hosts can be passthroughed.
      910B3/4
      Note: Only GPUs of this model installed on ARM hosts can be passthroughed.
      Hygon Z100, Z100L
      K100-AI
      Enflame S60
      Iluvatar CoreX Zhikai MR-V100
      Tiangai
      Vastai SV100, SG100
      Note: We do not recommend passthrough SG100 directly. It is recommended to slice it into vGPUs before use.
      KUNLUNXIN P800
      Alibaba PPU PPU-ZW810E
      Other MetaX N100, MOORE THREADS, Cambricon, and others
    • GPU Virtualization (vGPU)
      Through GPU virtualization technology, a single physical GPU is sliced into multiple smaller vGPUs, forming a fine-grained vGPU resource pool. This allows users to quickly create lightweight VM instances, achieve elastic resource deployment, effectively improve GPU utilization, and reduce costs.
      Note: Currently, deploying AI services on VM instances using vGPU is not supported.
      The following GPU models support virtualization slicing:
      Vendor Model
      NVIDIA Nvidia RTX 6000Ada, Nvidia RTX A6000
      Quadro RTX 8000, Quadro RTX 6000
      H100, H200, H800, H20
      Note: Supported only by ZStack Cloud H84R ISO.
      Nvidia L40, Nvidia L20, Nvidia L4
      Nvidia A40, Nvidia A16, Nvidia A10, Nvidia T4
      Tesla V100, Tesla P4/6/40/100, M6/10/60
      AMD FirePro S7150, FirePro S7150X2
      Vastai SV100, SG100
    • Dynamic Slicing and Scheduling (dGPU)
      Based on CUDA API interception and forwarding technology, physical GPU VRAM is sliced on demand and dynamically allocated to multiple VM instances, maximizing resource utilization. This is suitable for AI inference, deep learning, and other business scenarios.
      Note: To use this feature, you need to purchase the dGPU Slicing license.
      The following GPU models support the dGPU feature:
      Vendor Model
      NVIDIA Nvidia RTX 6000Ada, Nvidia RTX A6000
      GeForce RTX 5090, Geforce RTX 4090, Nvidia RTX 3090
      Quadro RTX 8000, Quadro RTX 6000
      H100, H200, H800, H20
      Note: Supported only by ZStack Cloud H84R ISO.
      Nvidia L40, Nvidia L20, Nvidia L4
      Nvidia A100, Nvidia A30
      Nvidia A40, Nvidia A16, Nvidia A10, Nvidia T4
      Tesla V100
  • Container
    • Whole-Card Scheduling

      Attach a physical GPU device directly to a container for use.

      The following GPU models support whole-card scheduling for containers:
      Vendor Model
      NVIDIA Nvidia RTX 6000Ada, Nvidia RTX A6000
      GeForce RTX 5090, Geforce RTX 4090, Nvidia RTX 3090
      Quadro RTX 8000, Quadro RTX 6000
      M4000, P2000
      GTX 1650/1660, GTX 1060ti
      H100, H200, H800, H20
      Nvidia L40, Nvidia L20, Nvidia L4
      Nvidia A100, Nvidia A30
      Nvidia A40, Nvidia A16, Nvidia A10, Nvidia T4
      Tesla V100, Tesla P4/6/40/100, M6/10/60
      K6000
      Huawei Atlas 300i pro
      Note: Supported only by Container Management on Kylin10SP3 2403.
      910B3/4
      Note: Supported only by Container Management on ARM-Kylin10SP3 2403.
      Hygon Z100, Z100L
      Note: Supported only by Container Management on x86-Kylin10SP3 2403.
      K100-AI
      Note: Supported only by Container Management on x86-Kylin10SP3 2403.
      Enflame S60
      Note: Supported only by Container Management on Kylin10SP3.
      Iluvatar CoreX Zhikai MR-V100
      Note: Supported only by Container Management on H84R and x86-Kylin10SP3 2403.
      Tiangai
      Note: Supported only by Container Management on H84R and x86-Kylin10SP3 2403.
    • VRAM Slicing

      Slice and isolate VRAM on single or multiple GPUs on a physical host, and allocate VRAM to containers on that host. VRAM slicing is supported only on a single GPU.

      The following GPU models support VRAM slicing for containers:
      Vendor Model
      NVIDIA Nvidia RTX 6000Ada, Nvidia RTX A6000
      GeForce RTX 5090, Geforce RTX 4090, Nvidia RTX 3090
      Quadro RTX 8000, Quadro RTX 6000
      M4000, P2000
      GTX 1650/1660, GTX 1060ti
      H100, H200, H800, H20
      Nvidia L40, Nvidia L20, Nvidia L4
      Nvidia A100, Nvidia A30
      Nvidia A40, Nvidia A16, Nvidia A10, Nvidia T4
      Tesla V100, Tesla P4/6/40/100, M6/10/60
      K6000
      Huawei Atlas 300i pro
      Note:
      • Slicing is supported only on a single GPU.
      • Supported only by Container Management on Kylin10SP3 2403.
      910B3/4
      Note:
      • Slicing is supported only on a single GPU.
      • Supported only by Container Management on ARM-Kylin10SP3 2403.
      Hygon Z100, Z100L
      Note:
      • Slicing is supported only on a single GPU.
      • Supported only by Container Management on x86-Kylin10SP3 2403.
      K100-AI
      Note:
      • Slicing is supported only on a single GPU.
      • Supported only by Container Management on x86-Kylin10SP3 2403.
      Iluvatar CoreX Zhikai MR-V100
      Note:
      • Slicing is supported only on a single GPU.
      • Supported only by Container Management on H84R and x86-Kylin10SP3 2403.
      Tiangai
      Note:
      • Slicing is supported only on a single GPU.
      • Supported only by Container Management on H84R and x86-Kylin10SP3 2403.

Inventory

Properties

Name Description Optional Valid Values Starting Version
uuid The UUID. For more information, see Resource Property. 5.1.8
chooser 5.1.8
description The description. For more information, see Resource Property. 5.1.8
deviceId The device ID. 5.1.8
hostUuid The host UUID. 5.1.8
isDriverLoaded Whether the driver is loaded. 5.1.8
memory The memory. 5.1.8
name The name. For more information, see Resource Property. 5.1.8
pciDeviceAddress The PCI device address. 5.1.8
power The power. 5.1.8
serialNumber The serial number. 5.1.8
status The status. 5.1.8
subvendorId The sub vendor ID. 5.1.8
type The type. 5.1.8
virStatus The virtualization status. 5.1.8
device The device 5.1.8
groupBy 5.1.8
iommuGroup The IOMMU group. 5.1.8
parentUuid The parent UUID. 5.1.8
pciSpecUuid The PCI spec UUID. 5.1.8
rev The device revision. 5.1.8
state The state. 5.1.8
subdeviceId The sub device ID. 5.1.8
vendorId The vendor ID. 5.1.8
vmInstanceUuid The VM instance UUID. 5.1.8
createDate The creation date. For more information, see Resource Property. 5.1.8
lastOpDate The last operation date. For more information, see Resource Property. 5.1.8
userTags The user tags. For more information, see CreateUserTag. The resource type is ZoneVO. 是 5.1.8
systemTags The system tags. For more information, see CreateSystemTag. The resource type is ZoneVO. 是 5.1.8
Sample
{
    "inventories": [
        {
            "chooser": "None",
            "createDate": "Jul 15, 2024 3:54:32 PM",
            "description": "3D controller: NVIDIA TU102GL [Quadro RTX 6000/8000]",
            "device": "TU102GL [Quadro RTX 6000/8000] [1e78]",
            "deviceId": "1e78",
            "hostUuid": "363c8eab666547e29a7a74bbde9c1967",
            "iommuGroup": "363c8eab666547e29a7a74bbde9c1967_/sys/kernel/iommu_groups/39",
            "isDriverLoaded": true,
            "lastOpDate": "Jul 17, 2024 4:03:05 PM",
            "mdevSpecRefs": [],
            "memory": 48318382080,
            "metaData": {},
            "name": "NVIDIA_TU102GL [Quadro RTX 6000/8000]_4885da63",
            "pciDeviceAddress": "0000:17:00.0",
            "pciSpecUuid": "016a96fd5aa648b290d2e96bf49f7407",
            "power": 250,
            "serialNumber": "1325119076420",
            "state": "Enabled",
            "status": "System",
            "subdeviceId": "13d8",
            "subvendorId": "10de",
            "type": "GPU_3D_Controller",
            "uuid": "4885da636aba4e12afcc5901e12673a3",
            "vendor": "NVIDIA",
            "vendorId": "10de",
            "virtStatus": "SRIOV_VIRTUALIZABLE"
        }
    ],
    "success": true
}

matchedPciDeviceOffering inventory

Name Description Optional Valid Values Starting Version
uuid The UUID. For more information, see Resource Property. 5.1.8
description The description. For more information, see Resource Property. 5.1.8
deviceId The device ID. 5.1.8
ramSize 5.1.8
name The name. For more information, see Resource Property. 5.1.8
subvendorId The sub vendor ID. 5.1.8
type The type. 5.1.8
groupBy 5.1.8
subdeviceId The sub device ID. 5.1.8
vendorId The vendor ID. 5.1.8
subvendorId The sub vendor ID. 5.1.8
createDate The creation date. For more information, see Resource Property. 5.1.8
lastOpDate The last operation date. For more information, see Resource Property. 5.1.8
userTags The user tags. For more information, see CreateUserTag. 是 5.1.8
systemTags The system tags. For more information, see CreateSystemTag. 是 5.1.8

PciDeviceSpec inventory

Name Description Optional Valid Values Starting Version
uuid The UUID. For more information, see Resource Property. 5.1.8
description The description. For more information, see Resource Property. 5.1.8
device The device. 5.1.8
isVirtual Whether the device is virtualized. 5.1.8
deviceId The device ID. 5.1.8
ramSize 5.1.8
romMd5sum 5.1.8
romVersion 5.1.8
name The name. For more information, see Resource Property. 5.1.8
subvendorId The sub vendor ID. 5.1.8
type The type. 5.1.8
groupBy 5.1.8
subdeviceId The sub device ID. 5.1.8
vendorId The vendor ID. 5.1.8
subvendorId The sub vendor ID. 5.1.8
createDate The creation date. For more information, see Resource Property. 5.1.8
lastOpDate The last operation date. For more information, see Resource Property. 5.1.8
userTags The user tags. For more information, see CreateUserTag. 5.1.8
systemTags The system tags. For more information, see CreateSystemTag. 5.1.8

Operations

QueryGpuDevice

Queries GPU devices. For example,
QueryGpuDevice

Primitive Fields of Query

See GPU Device Inventory.

Nested and Expanded Fields of Query

Field Inventory Description Starting Version
host Host Inventory The host that the device resides on. 5.1.8
macthedPciDeviceOffering matchedPciDeviceOffering inventory The matched PCI device offering. 5.1.8
mdevSpec The mdev device specification. 5.1.8
matchedPciDeviceOfferingRef. The matched PCI devices offering reference. 5.1.8
parent 5.1.8
pciDeviceSpec PciDeviceSpec inventory The PCI device specification. 5.1.8
vmInstance VM Instance Inventory The VM instances that run on the host. 5.1.8
PreviousHygon SENextPrimary Storage
CLI Command Manual | 5.5.30 | ZStack Cloud · ZCF | ZStack Resource Center