Bare-metal GPUs
Whole servers, with no virtual layer. 87 providers on EveryNeocloud so far.
A bare-metal GPU server is a whole physical machine rented to one customer, with no hypervisor, the software layer that splits a server into virtual machines. The customer gets every GPU, all the memory and direct access to the network cards. Neoclouds such as CoreWeave built their business on bare metal, and hosts such as Hetzner, OVHcloud and Leaseweb have long rented dedicated servers.
Why AI teams want it
Large training jobs need every bit of speed, and a virtual layer can slow the links between GPUs and to the network. On bare metal, GPUs talk to each other through NVLink and to other servers through InfiniBand or RoCE, a fast Ethernet method, with nothing in between. Teams can also install their own drivers and tools. The trade-off is that the customer manages more of the system.
How it is sold
Bare-metal GPU servers are usually rented by the month or on reserved contracts, often as eight-GPU nodes. Many providers wrap bare metal in Kubernetes or a job scheduler such as Slurm, so customers get the speed of bare metal with tools to run jobs. Providers compete on how fast they can hand over healthy servers and how they handle failures.
Famous bare-metal GPUs providers
Still to be written: Leaseweb, Vultr.
Further reading
Nothing matches those filters yet. Take one off, or check back: new pages are written every day.
Photo credits on this page
The small pictures above come from Wikimedia Commons under free licences. Some are cropped to fit.
- CoreWeave: Wikideas1, CC0, source
- OVHcloud: Velvet, CC BY-SA 4.0, source