Pod
A standard building block of a cluster: a set number of racks and GPUs with their own network layer.
Also: scalable unit, block
Large clusters are assembled from repeated units. A pod, or scalable unit, is a group of racks, for example 256 or 1,024 GPUs, wired to its own leaf switches and tested together before being joined to the rest through spine switches. Google uses "pod" for a group of TPUs joined by its inter-chip network, which in recent generations reaches thousands of chips. Building in pods lets providers bring capacity online in stages and hand a whole pod to one customer.