Procedure

1. Energize and connect hardware.

   Check your hardware devices. Configure servers and network devices according to Figure 1.

2. Install operating systems.

   Install operating system for the two servers according to following steps:

   1. Check BIOS.

      After you energize a server, manually start the server and enter its BIOS, making sure that SR-IOV Support and Intel VT-d / AMD IOMMU are enabled.

   2. Burn the custom ISO into a USB drive.
   3. Make the server start from the USB drive.
   4. Install the operating system following the wizard.

      Configure partitioning and begin to install the operating system for the server as a management node. The installation is an automatic process. During this process, set your root account and password. After the installation is finished, restart the management node and unplug the USB drive.

3. Configure the management network.
   Run following commands on the two management nodes in turn to configure the management network according to the table Table 1 in turn.

   # Take management node1 as an example.
   
   # Create the aggregate NIC bond0
   [root@localhost ~]# zs-bond-lacp -c bond0
   
   # Attach the two gigabit network interfaces, eth0 and eth1, of the management node1 to bond0.
   [root@localhost ~]# zs-nic-to-bond -a bond0 eth0
   [root@localhost ~]# zs-nic-to-bond -a bond0 eth1
   
   # After finishing the link aggregation configurations, configure the LACP aggregation on the corresponding vSwitch network interfaces.
   
   # Create the network bridge br_bond0 and set the IP address, netmask, and gateway of the management node1.
   [root@localhost ~]# zs-network-setting -b bond0 172.25.15.143 255.255.255.0 172.25.0.1
   
   # Check whether the aggregate port bond0 is created successfully.
   [root@localhost ~]# zs-show-network

4. Install HA suites.

   Install HA suites on either of the two management nodes to realize Multi-MN HA.

   Take management node1 as an example.

   1. Import HA suites to management node1 and unzip the files.
   2. Install HA suites on management node1.
   3. Use the VIP to access the UI of management node1 and finish the Cloud initialization.
   4. Switch the primary and secondary management nodes. Use the VIP to access the UI of management node2 and finish the Cloud initialization.
5. Install the license.

   Use the VIP to access the UI of either management node. Upload the Multi-MN license you prepared.

6. Create a cluster with Network Acceleration Support enabled.

   Use the VIP to access the UI of management node1. On the main menu of ZStack Cloud, choose Resource Center > Hardware > Computing Facility > Cluster. On the Cluster page, click Create Cluster. Then, the Create Cluster page is displayed.

   On the displayed page, set the following parameters:

   • Zone: The current zone is displayed.
   • Name: Enter a name for the cluster.
   • Description: Optional. Enter a description for the cluster.
   • Hypervisor: Choose KVM.
   • CPU Architecture: Choose x86_64.
   • Network Acceleration Support: Optional. Default: disabled. In this practice, enable this feature.
     Note: You can enable Network Acceleration Support for a cluster only when you create the cluster.
   • Network Configuration:
     • VDI Network: Optional. Leave it as blank in this practice.
     • Migration Network: Optional. Leave it as blank in this prectice.
   • Advanced:
     • VM CPU Model: Optional. Use the default setting in this practice.
     • Host CPU Model: Optional. Use the default setting in this practice.

   
   
   

7. Add the two management nodes to the cluster.

   On the main menu of ZStack Cloud, choose Resource Center > Hardware > Computing Facility > Host. On the Host page, click Add Host. Then, the Add Host page is displayed. Choose Manual Addition on the displayed page.

   Take management node1 as an example. Set the following parameters:

   • Name: Enter a name for the host.
   • Description: Optional. Enter a description for the host.
   • Virtualization Technology: Choose KVM.
   • Tag: Optional. Do not set this parameter in this practice.
   • Cluster: Choose the cluster you created with Network Acceleration enabled.
   • Host Info: Choose IP Address as the addition method.
     • IP Address: Enter the host IP. In this practice, enter the IP address of management node1: 172.25.15.143.
     • SSH Port: Enter the host port. Default: 22.
     • User Name: Enter the host user name. Default: root.
     • Password: Enter the password corresponding to the user name.
   • Scan Host IOMMU Setting: Optional. Default: disabled. In this practice, enable this feature.
     Note: The setting takes effect after you restart the host.
   • Intel EPT Hardware Assist: Optional. Default: enabled.
   • Add Another Host: Select this checkbox to add management node2.

   
   
   

8. Virtualize the physical NIC.
   On the details page of the host, choose Associated Resource > Physical NIC, locate a Mellanox CX-5 smart NIC and click SR-IOV on its action list to virtualized the smart NIC into VF NICs of the specified number.
9. Create two L2 network whose Network Acceleration Mode are respectively Smart NIC and Standard. Attach the two L2 networks to the cluster.
   1. Create an L2 network whose Network Acceleration Mode is Smart NIC and attach it to the cluster.

      On the main menu of ZStack Cloud, choose Resource Center > Network Resource > L2 Network Resources > L2 Network. On the L2 Network page, click Create L2 Network. Then, the Create L2 Network page is displayed.

      On the displayed page, set the following parameter:

      • Zone: The current zone is displayed.
      • Name: Enter a name for the L2 network.
      • Description: Optional. Enter a description for the L2 network.
      • Type: Choose L2VlanNetwork.
      • Cluster: Optional. Choose the cluster you created with Network Acceleration Support enabled.
      • Network Acceleration Mode: Choose Smart NIC.
      • VLAN ID: Enter 2334 in this practice.
      • NIC Name: Enter the name of the L2 network device. In this practice, enter dpdkbond2.

      
      
      

   2. Create an L2 network whose Network Acceleration Mode is Standard and attach it to the cluster.
      On the Create L2 Network page, set the following parameters:

      • Zone: The current zone is displayed.
      • Name: Enter a name for the L2 network.
      • Description: Optional. Enter a description for the L2 network.
      • Type: Choose L2VlanNetwork.
      • Cluster: Optional. Choose the cluster you created with Network Acceleration Support enabled.
      • Network Acceleration Mode: Choose Standard.
      • VLAN ID: In this practice, enter 2200.
      • NIC Name: Enter the name of the L2 network device. In this practice, enter bond0.

      
      
      

10. Create a VPC vRouter and two VPC networks.
    1. Create a VPC vRouter.

       Upload the VPC vRouter image you prepared to Cloud and create a management network and vRouter offering according to Table 3.

       On the main menu of ZStack Cloud, choose Resource Center > Network Resource > vRouter > VPC vRouter. On the VPC vRouter page, click Create VPC vRouter. Then, the Create VPC vRouter page is displayed.

       On the displayed page, set the following parameters:

       • Name: Enter a name for the VPC vRouter.
       • Description: Optional. Enter a description for the VPC vRouter.
       • vRouter Offering: Choose a vRouter Offering.
       • Cluster: Optional. Specify a cluster for the host on which the VPC vRouter is to be started.
       • Storage Allocation Policy: Specify how the Cloud allocates a primary storage. The following two policies are supported:
         • System Allocation: The Cloud allocates a primary storage according to the preconfigured policy.
         • Custom: Select a primary storage as needed.
           • Primary Storage: Select a primary storage for the VPC vRouter.
       • Host: Optional. Specify a host to start the VPC vRouter. In this practice, choose the management node1.
       • Default IPv4: Optional. Leave this parameter as blank in this practice.
       • Assign Management Network IP: Optional. Leave this parameter as blank in this practice.
       • DNS: Optional. Leave this parameter as blank in this practice.
       • CPU Pinning: Optional. Leave this parameter as blank in this practice.

       
       
       

    2. Create the VPC network1.

       Create the VPC network1 according to Table 4.

       On the main menu of ZStack Cloud, choose Resource Center > Network Resource > L3 Network Resources > VPC Network. On the VPC Network page, click Create VPC Network. Then, the Create VPC Network page is displayed.

       On the displayed page, set the following parameters:

       • Name: Enter a name for the VPC network. Enter VPC Network1 in this practice.
       • Description: Optional. Enter a description for the VPC network.
       • L2 Network: Choose an L2 network (Network Acceleration Mode: Standard).
       • VPC vRouter: Optional. Choose a VPC vRouter.
       • Network Range Method: Choose CIDR.
         • CIDR: Enter 192.167.0.1/24.
         • Gateway: Optional. Enter 192.167.0.1 in this practice.
         • DHCP Service: Optional. Enabled by default.
         • IP Allocation Policy: Optional. Choose Random in this practice.
         • DHCP IP: Optional. Leave this parameter as blank in this practice.
       • DNS: Optional. Leave this parameter as blank in this practice.

       
       
       

    3. Create the VPC Network2.

       Create the VPC network2 according to Table 5.

       On the displayed page, set the following parameters:

       • Name: Enter a name for the VPC network. Enter VPC Network2 in this practice.
       • Description: Optional. Enter a description for the VPC network.
       • L2 Network: Choose an L2 network (Network Acceleration Mode: Smart NIC).
       • VPC vRouter: Optional. Choose a VPC vRouter.
       • Network Range Method: Choose CIDR.
         • CIDR: Enter 192.169.0.1/24.
         • Gateway: Optional. Enter 192.169.0.1 in this practice.
         • DHCP Service: Optional. Enabled by default.
         • IP Allocation Policy: Choose Random in this network.
         • DHCP: Optional. Leave this parameter as blank in this practice.
       • DNS: Optional. Leave this parameter as blank in this practice.

       
       
       

11. Create two VM instances. Attach each one with the two VPC networks you created before.

    On the main menu of ZStack Cloud, choose Resource Center > Resource Pool > > Virtual Resource > VM Instance. On the VM Instance page, click Create VM Instance. Then, the Create VM Instance page is displayed. Choose Fast Creation.

    On the displayed page, set the following parameters:

    • Name: Enter a name for the VM instance.
    • Quantity: Enter 2 in this practice.
    • Tag: Optional. Do not set tags for the VM instance in this practice.
    • Offering: Choose Basic Offering.
      • Instance Offering: Choose an instance offering.
      • Image: Choose an image.
      • Data Volume: Optional. No data volume is attached by default.
    • Network Configuration:
      • Network: Attach the VPC network1 and VPC network2 to the VM instance.
      • Make Default: Make the VPC network2 as the default network.
      • Enable SR-IOV: Optional. Disable SR-IOV in this practice.
      • Assign IP: Optional. The checkbox is not selected by default. The IP is randomly allocated by DHCP.
      • MAC Address: Optional. The checkbox is not selected by default. The MAC is randomly allocated by DHCP.
      • Security Group: Optional. Do not set security groups in this practice.
      • EIP: Optional. Do not set EIPs in this practice.
    • User Data: Optional. Do not set User Data in this practice.

    
    
    

    
    
    

12. Compare the VM vNIC performance with the vDPA NIC performance.
    In this practice, use an iperf3 tool to test the network performance of the two VM instances. Compare the vNIC performance and vDPA NIC performance from 3 aspects: bandwidth, packet loss rate, and throughput.
    1. Query NIC information of the two VM instance.

       Name	NIC Type	IP Address
       VM Instance1	vNIC	192.167.0.166
       	vDPA	192.169.0.214
       VM Instance2	vNIC	192.167.0.193
       	vDPA	192.169.0.253

       
       
       

       
       
       

    2. UDP small packet performance testing (packet size: 16 byte).

       Assume that the VM instance1 is the client and VM instance2 is the server.

       The testing involves two rounds.

       1. First Round: Send UDP packets from the VM instance2 (vNIC) to the VM instance1 (vNIC). View the bandwidth, packet loss rate, and throughput.
       2. Second Round: Send UDP packets from the VM instance2 (vDPA) to the VM instance1 (vDPA). View the bandwidth, packet loss rate, and throughput.

       # Install iperf3 on both the client and the server.
       [root@localhost ~]# yum install iperf3
       
       # First-Round Testing
       
       # Run the following command on the server:
       [root@localhost ~]# iperf3 -s
       
       # Run the following command on the client:
       [root@localhost ~]# iperf3 -u -l 16 -b 100m -t 30 -c 192.167.0.193 -i 1
       
       # iperf3 commands information
       # -s //Run in the server mode.
       # -u //UDP mode.
       # -l //UDP small packet size. Unit: byte.
       # -b //Target bandwidth. Unit: bits/sec.
       # -t //Testing period. Unit: s.
       # -c //Run in the client mode and specify the server IP.
       # -i //Echo interval. Unit: second.
       
       # The first-round testing lasts about 30 seconds.
       
       # Second-Round Testing
       
       # Run the following command on the client:
       [root@localhost ~]# iperf3 -u -l 16 -b 100m -t 30 -c 192.169.0.253 -i 1
       
       # The second-round testing lasts about 30 seconds.

       Test Result

       UDP Small Packet Testing	Server - Client	Bandwidth (Mbits/sec)	Packet Loss Rate (%)	Throughput (pps)
       First Round	VM Instance2 (vNIC) - VM Instance1 (VNIC)	12.1	0.46	95144.3
       Second Round	VM Instance2 (vDPA) - VM Instance1 (vDPA)	19.2	0.74	151418

       First Round
       
       

       Second Round
       
       

    3. UDP big packet performance testing (packet size: 1460 byte).

       Assume that the VM instance1 is a client and VM instance2 is a server.

       The testing involves two rounds:

       1. First Round: Send UDP packets from VM instance2 (vNIC) to VM instance1 (vNIC). View the bandwidth, packet loss rate, and throughput.
       2. Second Round: Send UDP packets from VM instance2 (vDPA) to VM instance1 (vDPA). View the bandwidth, packet loss rate, and throughput.

       # First-Round Testing
       
       # Run the folliwng command on the client (the UDP packet size is 1460 by default):
       [root@localhost ~]# iperf3 -u -b 1G -t 30 -c 192.167.0.193 -i 1
       
       # The first-round testing lasts about 30 seconds.
       
       # Second-Round Testing
       
       # Run the following command on the client (the UDP packet size is 1460 byte by default):
       [root@localhost ~]# iperf3 -u -b 1G -t 30 -c 192.169.0.253 -i 1
       
       # The second-round testing lasts about 30 seconds.

       Testing Result

       UDP Big Packet Testing	Server - Client	Bandwidth (Mbits/sec)	Packet Loss Rate (%)	Throughput (pps)
       First Round	VM Instance2 (vNIC) - VM Instance1 (vNIC)	267	51	94735.1
       Second Round	VM Instance2 (vDPA) - VM Instance1 (vDPA).	843	1.8	148343.5

       First Round
       
       

       Second Round