ZCube - A next-generation large-scale model inference network architecture jointly launched by Zhipu AI and Tsinghua University

What is ZCube?

ZCube is a next-generation large-model inference network architecture jointly developed by Zhipu AI, Yuhun Networks, and Tsinghua University. It is specifically designed to solve structural network congestion in separate PD deployments. The architecture eliminates traditional Spine layer switches, adopting a flat topology across the entire network and a single/multi-track hybrid access mechanism to achieve global traffic decoupling and discretized routing. In production environment testing with GLM-5.1 coding, ZCube reduces capital expenditure on switches and optical modules by 33% while maintaining the same GPU and software stack, increases average GPU inference throughput by 15%, and reduces TTFT P99 by 40.6%.

ZCube's main functions

• Flat networking with Spine layer removedBreaking away from the traditional hierarchical stacking approach of the Clos architecture, it retains the Leaf switch layer and interconnects them through a complete bipartite graph, compressing the network diameter from 3 hops to 2 hops.
• Single/Multi-track Hybrid Access MechanismOne set of Leaf switches connects GPUs with consecutive labels using a single-track method, while another set connects GPUs with the same labels using a multi-track method, thus naturally discretizing the dynamic asymmetric traffic generated by PD separation.
• Global load balancing routingZCube routing strategy ensures that there is only one optimal path between any two GPU pairs, avoiding traffic conflicts caused by multi-path routing, and achieving ideal load balancing across the entire network in both training and inference scenarios.
• Structural congestion eliminationBy innovating the architecture layer, the cross-node transmission traffic of KV Cache is globally decoupled and discretized, fundamentally eliminating local hotspot links and PFC backpressure.

ZCube's technical principles

• Complete bipartite graph topologyThe Leaf switches are divided into two groups based on parity. Switches within a group are not interconnected, while each switch in an inter-group is fully connected to all switches in the other group, forming a flat network.
• Dual-port differentiated accessEach GPU network card has two ports connected via a single-track (connecting to odd-numbered switches) and a multi-track (connecting to even-numbered switches) approach, respectively, using mathematical mapping formulas to achieve deterministic routing.
• Deterministic shortest path routingBased on the modular operation and rounding up mapping between GPU IDs and switch IDs, it is ensured that there is only one optimal path between any two GPUs that passes through two Leaf switches.
• Traffic pattern adaptationTo address the source-destination asymmetry and dynamically changing KV Cache transmission characteristics caused by PD separation, the topology structure itself achieves natural traffic dispersion, without relying on transport layer mechanisms such as adaptive routing or packet spraying.

How to use ZCube

• Architecture planningThe Leaf switches are divided into two groups according to their parity numbers, and a fully bipartite interconnect topology is designed, eliminating the traditional Spine layer.
• Port access configurationConfigure single-track and multi-track hybrid access policies for each GPU network card's dual ports and connect them to the corresponding odd/even switches.
• Automated deploymentUsing the ZCube controller, data center layout design tools, and connection correctness detection program, configurations are automatically generated and distributed in batches.
• Production ValidationBefore the inference service goes live, bandwidth, latency, and PFC events are monitored, and the flow is officially switched after confirming that there are no structural congestion hotspots.

ZCube's core advantages

• Costs significantly reducedCompared to Clos/ROFT, the Wanka Intelligent Computing Cluster reduces the investment in switches and optical modules by about 33%, saving approximately 210 million to 640 million yuan in network hardware investment.
• Improved inference performanceIn production testing with GLM-5.1 coding, the average GPU inference throughput increased by more than 15%, and the P99 percentile of TTFT decreased by 40.6%.
• Super scalabilityBased on the existing 51.2T switch, a flat network connecting 16,384 400Gbps network cards can be built, and through multi-plane partitioning, it can support tens of thousands to hundreds of thousands of GPU interconnection.
• Zero-intrusive upgradeWithout modifying the GPU hardware, software stack, or application logic, the potential of existing hardware can be unlocked simply by optimizing the network architecture layer.

ZCube's project address

• Project official websitehttps://z.ai/blog/zcube

ZCube's Competitive Product Comparison

ZCube Application Scenarios

• Ultra-large-scale LLM inference clusterIt is suitable for 1,000 to 10,000-card inference clusters with separate PD deployment, and solves the network bottleneck caused by cross-node transmission of KV Cache.
• Long Context Reasoning ServiceThe architecture can effectively alleviate the constraints of network bandwidth on the latency of the first token and the overall throughput in long sequence scenarios.
• High-density intelligent computing centerIt provides a highly balanced, low-latency, and high-bandwidth-utilization network foundation for mixed training and inference workloads.
• MaaS cloud service platformThe architecture can reduce the overall cost of inference services and improve tail latency stability in multi-tenant high-concurrency scenarios.