Training clusters
Reserved clusters of thousands of GPUs on one fabric. 105 providers on EveryNeocloud so far.
A training cluster is a block of hundreds to hundreds of thousands of GPUs joined on one fast network, reserved by one customer for months or years. This is how frontier models are trained, and it is the core business of the largest neoclouds. Deals are signed as contracts worth billions of dollars, often before the data centre is built.
What makes a cluster
Inside each server, GPUs talk over NVLink; between servers they use InfiniBand or fast Ethernet in a layout called a fat tree, so any GPU can reach any other at full speed. Shared storage feeds training data, and software such as Slurm or Kubernetes schedules jobs and restarts them when a GPU fails, which in a cluster of 100,000 GPUs happens many times a day.
How it is sold
Customers sign take-or-pay contracts: they pay for the capacity whether or not they use it. Microsoft, OpenAI, Meta and Google have signed such deals with CoreWeave, Nebius, Nscale, IREN and others. The contracts let providers borrow against them to buy GPUs. Critics say this ties a small number of buyers and lenders together; providers say long contracts make the business stable.
Famous training clusters providers
Further reading
馃嚥馃嚲 Malaysia and 4 moreYTL AI Cloud
Malaysian AI cloud and supercomputer run from a solar-backed Johor data centre parkNeocloud 路 Nvidia Blackwell 路 2024
馃嚫馃嚞 Singapore and 4 moreMegaspeed
Singapore AI cloud firm investigated over Nvidia chip diversion to ChinaNeocloud 路 Nvidia Blackwell 路 2015
馃嚘馃嚭 Australia and 2 moreFirmus
Neocloud operator building immersion-cooled AI Factories across Asia-PacificNeocloud 路 Nvidia Blackwell
馃嚭馃嚫 United StatesArc Compute
Turnkey NVIDIA GPU cluster design, deployment and management for AI workloadsNeocloud 路 Nvidia Blackwell 路 2018
WorldGMI Cloud
AI-native cloud for production inference, GPU clusters and agent runtimesNeocloud 路 Nvidia Hopper
馃嚭馃嚫 United StatesLambda
AI-only GPU cloud running NVIDIA supercomputers for training and inferenceNeocloud 路 Nvidia Blackwell 路 2012
Photo credits on this page
The small pictures above come from Wikimedia Commons under free licences. Some are cropped to fit.
- CoreWeave: Wikideas1, CC0, source
- YTL AI Cloud: YTL Power International Berhad, CC BY-SA 4.0, source