Issue

After you modify VPC firewall rules, change rule order, or reconnect a VPC, a VPC router that uses a VyOS image can experience rule-delivery timeouts, mismatches between platform rules and applied router rules, or service network connectivity failures. Use this article to confirm the image type, configuration scale, management-node tasks, VyOS execution logs, and applied rules before selecting a remediation path.

Environment

• Product: ZStack Cloud.
• Component: Cloud.
• Versions: 5.4.x and 5.5.x.
• Affected objects: VPC routers, VyOS VPC images, VPC firewalls, EIPs, port forwarding, load balancers, and the iptables delivery path.
• Deployment mode: A single VPC router or VPC high availability environment on KVM. The risk is higher when the VPC has many subnets, EIPs, port-forwarding rules, load balancers, or firewall rules.

Symptoms

• A rule-delivery task fails, times out, or does not complete for an extended period after you modify firewall rules, change rule order, or reconnect a VPC.
• The UI or API shows that firewall rule sets and rules have been saved, but the VyOS configuration or iptables rules on the VPC router are missing.
• Management-node logs contain firewall synchronization, /fw/apply/rule, /fw/apply/ruleSet/changes, fail to commit, return code -1, or task-timeout records.
• The VPC router zvr.log records a firewall-chain configuration failure for a service NIC, such as eth5.in, or records removal of old rules without corresponding rule recreation.
• Traffic across VPC subnets, through EIPs, port forwarding, load balancers, or the VPC firewall is interrupted. Brief interruptions can also occur while the VPC reconnects or rules are delivered.

Possible causes

• VyOS CLI execution time increases as the existing configuration grows. COMMIT operations with large configurations are more likely to time out on ARM or lower-performance CPUs.
• A single VPC router has many attached subnets, EIPs, port-forwarding rules, load balancers, and firewall rules, so a single change triggers many configuration commands.
• A COMMIT exception or timeout occurs after old rules are removed during synchronization, preventing new rules from being recreated and causing the applied rules to differ from the expected platform state.
• A VyOS CLI exception causes configuration commits to fail.
• Primary and secondary VPC routers in a high availability environment are not refreshed together, so failover uses a router with incomplete rules.
• Routing, NAT, security groups, NICs, VLAN/VXLAN, public networks, or application services can also cause connectivity failures.

Troubleshooting steps

1. Confirm the image type and configuration scale

Confirm the image type used by the affected VPC, the VPC router status, the number of networks, the number of firewall rule sets, and the total rule count. Check these items on the VPC router details page, or run the following commands:

QueryVirtualRouterVm uuid=<VPC_ROUTER_UUID>
QueryFirewallRuleSet firewallUuid=<VPC_FIREWALL_UUID>
QueryFirewallRule ruleSetUuid=<FIREWALL_RULE_SET_UUID>

Confirm that the VPC router status is Running or Connected. If the environment has no established performance threshold, treat a router with more than 20 attached networks or hundreds to thousands of rules as a large configuration.

If the rule count is low, the image is not VyOS, or the configuration is not large, prioritize routing, NAT, security-group, and other network-failure checks.

2. Check management-node tasks and logs

Check management-node logs during the failure window:

grep -E 'ReconnectVirtualRouter|/fw/apply/rule|/fw/apply/ruleSet/changes|fail to commit|return code -1|OperationTimeout|TimeoutException' /var/log/zstack/manager.log

If logs have rotated, also check the archived logs for the relevant date. Record the API task start and end times, request ID, VPC router UUID, and firewall UUID.

When firewall synchronization succeeds, the logs do not contain fail to commit, return code -1, timeouts, or exception stack traces. If an exception occurs during bulk removal, COMMIT, or rule-set changes, continue by checking the applied router rules.

3. Compare VyOS configuration, iptables rules, and execution logs

Log in to the VPC router through a supported operations entry point and run the following checks:

sudo grep -E 'eth[0-9]+\.(in|out|local)|<FIREWALL_RULE_NAME>|fail to commit|return code -1' /home/vyos/zvr/zvr.log
show firewall name <FIREWALL_RULE_NAME>
sudo iptables-save | grep -E 'eth[0-9]+\.(in|out|local)|<FIREWALL_RULE_NAME>'

For VyOS CLI delivery, compare the show firewall output with the expected platform rules. For iptables delivery, compare the iptables-save output. Check rule chains, actions, addresses, ports, and order instead of relying only on the rule count in the UI.

If eth*.in, eth*.out, or a specified rule set fails to configure, if logs show removal without recreation, or if applied rules are missing, continue with the VyOS delivery inconsistency or performance-bottleneck path.

4. Isolate the scope

• If a VyOS image, a large configuration, management-node timeouts, and missing router rules occur together, evaluate image replacement or the iptables delivery path in the next section.
• If platform and router rules are consistent but traffic remains unavailable, investigate routing, NAT, security groups, public networks, load-balancer backends, or the application path. Do not repeatedly reconnect the VPC because this can extend the interruption window.
• If only the primary router has correct rules in a VPC high availability environment, check the secondary router and refresh both routers by using the method supported by the current version.
• If only one rule set or NIC is abnormal, perform a supported firewall refresh or VPC reconnection during a service change window. Record rule and traffic status before the operation.

Resolution

Recommended: Upgrade or replace the VPC image with an Euler image

Prioritize the latest Euler VPC image supported by the current version to reduce dependence on large VyOS CLI COMMIT operations. During a service change window, use the operating-system replacement, image replacement, upgrade, migration, or rebuild workflow supported by the current version. After the change, reconnect or start the VPC and verify firewall rules, EIPs, port forwarding, load balancers, and routes individually.

Temporary option: Enable iptables firewall delivery for a specified VPC

If the image cannot be replaced, you can enable iptables delivery for a specified VPC router after confirming that the current version supports it:

UpdateResourceConfig category=vpc name="configure.firewall.with.iptables" resourceUuid=<VPC_ROUTER_UUID> value=true
ReconnectVirtualRouter vmInstanceUuid=<VPC_ROUTER_UUID>

For VPC high availability, refresh the primary and secondary routers according to the product behavior of the current version, and confirm that rule synchronization completes on both routers.

The category, resource binding, and object used by resourceUuid can differ between versions. Before proceeding, use QueryResourceConfig or the API documentation for the current version to confirm that the configuration exists and that the parameters apply. If the API reports that the configuration does not exist or that the parameters are invalid, do not trial parameters in a production environment.

Verification

After remediation, repeat the preceding checks and confirm the following:

• The platform rule count matches the applied rules on the VPC router.
• Added, removed, and reordered rules all take effect correctly.
• Management-node logs no longer report fail to commit, return code -1, or firewall synchronization timeouts.
• Traffic across subnets, EIPs, port forwarding, load balancers, and firewall allow or deny rules behaves as expected.
• Primary and secondary rules are consistent in a VPC high availability environment, and network connectivity remains normal after failover.

You can check the logs again:

grep -E '/fw/apply/rule|/fw/apply/ruleSet/changes|fail to commit|return code -1' /var/log/zstack/manager.log
sudo grep -E 'fail to commit|return code -1|<FIREWALL_RULE_NAME>' /home/vyos/zvr/zvr.log

Risks and rollback

• Rule delivery, VPC reconnection, image replacement, or primary-secondary refresh can briefly interrupt network connectivity. Perform these operations during a service change window.
• With a large VPC configuration, repeated reconnection can extend the interruption time or expand the impact scope.
• Record the original ResourceConfig value before enabling iptables delivery. To roll back, restore the configuration to false, refresh the VPC router by using the method supported by the current version, and verify that rules are restored.
• Follow the current-version workflow for rolling back image replacement, upgrades, migration, or rebuilds. Confirm backups, the service window, and rollback conditions before proceeding.

Information to collect

If the issue persists after remediation, collect the following information:

• Management-node logs, zvr.log, iptables-save, and show firewall output covering the full window before and after the failure.
• Accurate timestamps for the rule change, VPC reconnection, failure onset, recovery, and log collection.
• VPC router UUID, firewall UUID, rule-set UUID, affected NIC or IP address, traffic five-tuple, image type, and Cloud version.
• Counts of VPC subnets, EIPs, port-forwarding rules, load balancers, firewall rule sets, and firewall rules.
• Whether the iptables ResourceConfig is enabled and the command result.