Cluster
Many servers joined by a fast network and shared storage so they can work together on one job.
Also: GPU cluster, training cluster
A cluster is a group of computers that act together. An AI training cluster joins hundreds to hundreds of thousands of GPUs with a low-delay network, shares a parallel file system and runs a job scheduler that hands out work. The largest, such as xAI's Colossus or the clusters built for OpenAI, Meta and Anthropic, hold over 100,000 accelerators and draw hundreds of megawatts. Clusters are usually built in blocks of a set size, such as a pod or scalable unit, and their usefulness depends as much on the network and on reliability as on the GPU count.