Introduction

ZStack API Router is the unified API gateway for large model services in the ZStack AI platform. It provides model access, route governance, permission control, quota management, audit tracing, usage statistics, and health observation for administrators and platform users.

This white paper focuses on product capabilities, architecture, key mechanisms, and deployment practices. It is intended to help architects, administrators, and operations and maintenance engineers understand the product boundary and design a stable production deployment.

Product Version

This document applies to ZStack API Router V1.0 and later compatible versions. If product behavior differs across versions, use the release notes and delivery documents of the target version as the authoritative reference.

Intended Audience

This document is intended for:

• Architects responsible for AI platform gateway design.
• Platform administrators responsible for model access and route governance.
• Operations and maintenance engineers responsible for deployment, monitoring, and upgrade.
• Security administrators responsible for credential, audit, and permission policies.

Product Overview

ZStack API Router sits between business applications and upstream model services. It provides a unified invocation entry and governance layer for different model providers, deployment forms, and organizations.

Core Value

Typical values include:

• Unified model invocation protocol for business systems.
• Centralized model, channel, and Route Policy management.
• Organization, user, and model-level access control.
• Quota and usage statistics for cost management.
• Audit Log and Usage Log records for troubleshooting and compliance.
• Health status and metrics for operations and maintenance visibility.
• LobsterPool-compatible entry for AI platform integration.

Functional Architecture

The functional architecture of ZStack API Router covers unified access, gateway governance, organization and permission, operations and observability, and platform lifecycle capabilities.

These capabilities allow administrators to manage model invocation entries, tokens, model visibility, channels, Route Policies, usage metering, audit records, system settings, export tasks, and OpenAPI contracts from one gateway layer.

Overall Architecture

The following figure shows the relationship between ZStack API Router and related components.

The major components are:

Software Architecture

The software architecture separates access entry points, the unified entry layer, the ZStack API Router service layer, upstream model services, and shared data and infrastructure services.

Business applications, agent applications, and the admin console access the gateway through OpenAI-compatible APIs or management APIs. The service layer handles OpenAPI requests, request forwarding, channel adaptation, health probing, model synchronization, shared policy enforcement, usage records, audit redaction, settings, and telemetry.

Deployment Modes

Standalone Mode

Single-node deployment places all gateway components on one host. It is suitable for test, demonstration, POC, and small-scale environments.

Advantages:

• Simple deployment and maintenance.
• Small resource footprint.
• Suitable for validating model access and Route Policy quickly.

Limitations:

• Application restart may interrupt model invocation.
• Availability depends on a single host and a single database instance.
• Not recommended for key production workloads.

Production Entry Mode

Production deployment should use multiple gateway instances behind Nginx, Ingress, or a load balancer. PostgreSQL should use a production-grade backup and high-availability strategy.

Recommended practices:

• Deploy at least two zr-server instances.
• Deploy zr-worker separately when asynchronous task load is high.
• Use an external PostgreSQL service with scheduled backup.
• Configure health checks and readiness checks at the entry layer.
• Use TLS and a stable domain name for external access.
• Collect metrics and logs into a unified monitoring system.

LobsterPool-Compatible Mode

The LobsterPool-compatible mode is suitable when ZStack API Router is used as an AI platform model-service entry. It keeps compatibility with the existing platform invocation path while centralizing Route Policy, quota, audit, and usage statistics in the gateway.

Model Access and Channel Management

ZStack API Router abstracts upstream model services as channels and models.

Resource Architecture

ZStack API Router resources are organized into the identity and organization domain, model service governance domain, and operations and audit domain.

The identity and organization domain defines users, organizations, role sets, and model access rules. The model service governance domain defines tokens, channels, channel models, and Route Policies. The operations and audit domain records Usage Log records, Audit Log records, budgets, quotas, and settings.

Channel Lifecycle

A channel records the upstream endpoint, authentication method, timeout, retry strategy, and health state. Administrators can configure one or more channels for different upstream model services.

Channel management should follow these principles:

• Use a dedicated credential for each upstream service.
• Configure timeout and retry based on the upstream service SLA.
• Enable health checks before allowing production traffic.
• Avoid sharing one channel across unrelated environments.

A model represents a callable model exposed by the gateway. A model may point to one or more channels and can be enabled or suspended independently. Model management should follow these principles:

• Use stable model names for business integration.
• Keep a separate test model before enabling production traffic.
• Suspend models that have no owner or no valid upstream channel.
• Record model capability, cost level, and applicable scenario.

Data-Plane Invocation Flow

When a business application calls a model, the gateway processes the request through the following path:

1. Receive the request from the OpenAI-compatible or LobsterPool-compatible entry.
2. Authenticate the API Key generated by the token.
3. Identify tenant, organization, user, and requested model.
4. Check model access permission and quota.
5. Match Route Policy and select an upstream channel.
6. Forward the request to the upstream model service.
7. Return the response to the caller.
8. Write Usage Log, Audit Log, and metric data.

This flow ensures that model invocation can be controlled, traced, and measured consistently.

Main Supported Interfaces

Route Governance

Route Policy controls how traffic is sent to upstream model services. It can be designed by model, organization, user, channel priority, health status, quota, or business scenario.

Common policies include:

• Primary and secondary channels: use the primary channel first and switch to the secondary channel when the primary channel is unhealthy.
• Organization-based routing: send different organizations to different upstream services.
• Model-based routing: bind specific models to specific channels.
• Grayscale routing: route a small percentage of traffic or selected organizations to a new channel.
• Cost-aware routing: route low-priority traffic to lower-cost models or channels.

Route governance should have a clear verification method. After changing a policy, administrators should use test traffic to confirm actual route results and then observe Usage Log and Audit Log.

Security and Permission

ZStack API Router uses credential authentication, organization isolation, model access control, quota policy, and Audit Log to protect model services.

Credential Protection

Tokens are the credential management objects used by callers to access the gateway. After a token is created, the system generates API Key plaintext for the caller. Administrators should:

• Assign independent credentials to different applications or organizations.
• Set expiration time and owner for each credential.
• Suspend unused or leaked tokens immediately.
• Avoid storing plaintext credentials in scripts, images, or public repositories.

Access Control

Access control can be configured by user, organization, and model. Before allowing a business system to call production models, administrators should confirm:

• The caller belongs to the expected tenant or organization.
• The model is visible and enabled for that organization.
• The Route Policy does not send traffic to unauthorized upstream services.
• The quota policy matches the business budget.

Upstream Address Security

Administrators should configure only trusted upstream model service addresses and avoid pointing channels to unknown internal addresses. The entry proxy and gateway should both restrict unnecessary outbound access.

Quota and Metering

Metering Dimensions

Quota controls request count, token usage, or cost budget. Usage statistics help administrators analyze model consumption and allocate cost.

Recommended quota practices:

• Set independent quotas for production, test, and development organizations.
• Configure smaller quotas for new applications during trial.
• Review usage by organization, model, and credential periodically.
• Use alerts or manual review for abnormal traffic growth.

Usage Log records should contain at least:

• Request time.
• Tenant and organization.
• Credential or caller identity.
• Requested model.
• Upstream channel.
• Token usage.
• Latency.
• Result status and error reason.

Quota Governance

Quota should be configured by tenant, organization, user, model, or credential according to management requirements. Production quotas should be reviewed periodically and adjusted based on usage evidence.

Audit and Observability

Usage Log

Usage Log records model invocation and consumption information. It is used for cost analysis, quota verification, and request troubleshooting.

Audit Log

Audit Log records key operations and invocation decisions. It helps administrators answer who changed credentials, channels, models, Route Policies, and which route decision was used by a request.

Health and Metrics

ZStack API Router provides health status, metrics, logs, Usage Log, and Audit Log. These signals should be used together.

Production monitoring should include:

• Gateway request success rate and error rate.
• Upstream model service error rate.
• P95 and P99 latency.
• Database connection status.
• Worker component task backlog.
• Disk usage of log and backup directories.
• Expiring API Keys or certificates.

High Availability and Grayscale Practices

For production environments, use multiple gateway instances and avoid single points of failure.

Recommended design:

• Multiple zr-server instances behind an entry layer.
• Health checks configured on the entry layer.
• PostgreSQL with backup, monitoring, and recovery plan.
• Stateless application instances where possible.
• Centralized log and metric collection.

Grayscale release reduces upgrade and Route Policy change risk. Common grayscale methods include:

• Route one organization or one tenant to the new version first.
• Use a dedicated domain name for validation traffic.
• Configure a small traffic percentage at the entry layer.
• Keep the old version and old Route Policy available during observation.

During grayscale, monitor Usage Log, Audit Log, latency, error rate, and upstream channel health continuously.

Operations and Maintenance Recommendations

• Manage production configuration by environment and avoid mixing test and production values.
• Keep credentials and private keys outside the image package.
• Record each change to channel, model, Route Policy, and quota.
• Back up configuration before upgrade.

Appendix: Operations Reference