Rail-optimised network
A cluster wiring scheme where GPU number one in every server joins one switch group, GPU two another, and so on.
Also: rail-optimized
In an eight-GPU server each GPU has its own network card. A rail-optimised design connects all the first cards of every server to one set of leaf switches, all the second cards to another, and so on, creating eight rails. Because training libraries mostly exchange data between GPUs in the same position, most traffic stays on one rail and needs fewer switch hops. Nvidia recommends this layout in its reference architectures, and most large InfiniBand and Ethernet AI clusters use some form of it.