News
SovereignDeutsche Telekom boosts its sovereign AI push with 100 million for MistralInferenceAMD Completes Taalas Acquisition to Strengthen AI Inference PortfolioSovereignIndia’s Sarvam AI launches GPU service for open modelsClustersNew UB North Campus structure to house AI supercomputer system for research usesClustersIntroducing Together Storage Accelerator: fast, free data caching for your GPU clusterNeocloudsJensen Huang’s Charity Committed $350 Million to CoreWeaveClustersOak Ridge's AI Factory Moment: Lux, Nuclear Power, and the Next Data Center FrontierDealsNscale’s ByteDance GPU deal draws scrutiny over US chip curbsDealsNscale’s ByteDance Deal Faces Scrutiny Under US Chip CurbsSovereignNexusIgnite Plans Sovereign AI Cloud in Jordan, Starting With Its UniversitiesSovereignDeutsche Telekom boosts its sovereign AI push with 100 million for MistralInferenceAMD Completes Taalas Acquisition to Strengthen AI Inference PortfolioSovereignIndia’s Sarvam AI launches GPU service for open modelsClustersNew UB North Campus structure to house AI supercomputer system for research usesClustersIntroducing Together Storage Accelerator: fast, free data caching for your GPU clusterNeocloudsJensen Huang’s Charity Committed $350 Million to CoreWeaveClustersOak Ridge's AI Factory Moment: Lux, Nuclear Power, and the Next Data Center FrontierDealsNscale’s ByteDance GPU deal draws scrutiny over US chip curbsDealsNscale’s ByteDance Deal Faces Scrutiny Under US Chip CurbsSovereignNexusIgnite Plans Sovereign AI Cloud in Jordan, Starting With Its Universities
EveryNeocloud

Bare-metal GPUs

Whole servers, with no virtual layer. 88 providers on EveryNeocloud so far.

A bare-metal GPU server is a whole physical machine rented to one customer, with no hypervisor, the software layer that splits a server into virtual machines. The customer gets every GPU, all the memory and direct access to the network cards. Neoclouds such as CoreWeave built their business on bare metal, and hosts such as Hetzner, OVHcloud and Leaseweb have long rented dedicated servers.

Why AI teams want it

Large training jobs need every bit of speed, and a virtual layer can slow the links between GPUs and to the network. On bare metal, GPUs talk to each other through NVLink and to other servers through InfiniBand or RoCE, a fast Ethernet method, with nothing in between. Teams can also install their own drivers and tools. The trade-off is that the customer manages more of the system.

How it is sold

Bare-metal GPU servers are usually rented by the month or on reserved contracts, often as eight-GPU nodes. Many providers wrap bare metal in Kubernetes or a job scheduler such as Slurm, so customers get the speed of bare metal with tools to run jobs. Providers compete on how fast they can hand over healthy servers and how they handle failures.

Famous bare-metal GPUs providers

Still to be written: Leaseweb, Vultr.

Further reading

🇺🇸 United States and 1 moreCoreWeave US GPU cloud running AI data centres for hyperscalers and model labsNeocloud · Nvidia Hopper · 2017 Megaspeed 🇸🇬 Singapore and 4 moreMegaspeed Singapore AI cloud firm investigated over Nvidia chip diversion to ChinaNeocloud · Nvidia Blackwell · 2015 Polarise 🇩🇪 GermanyPolarise German sovereign AI cloud running its own data centres and open-weight model APIsNeocloud · Nvidia Blackwell · 2024 Firmus 🇦🇺 Australia and 2 moreFirmus Neocloud operator building immersion-cooled AI Factories across Asia-PacificNeocloud · Nvidia Blackwell Arc Compute 🇺🇸 United StatesArc Compute Turnkey NVIDIA GPU cluster design, deployment and management for AI workloadsNeocloud · Nvidia Blackwell · 2018 GMI Cloud WorldGMI Cloud AI-native cloud for production inference, GPU clusters and agent runtimesNeocloud · Nvidia Hopper NHN Cloud 🇰🇷 South KoreaNHN Cloud South Korean cloud provider with large GPU clusters for AI trainingNeocloud · Nvidia Blackwell WhiteFiber 🇺🇸 United StatesWhiteFiber AI data centres, private GPU clouds and reserved GPU capacity in North AmericaNeocloud · Nvidia Blackwell Crusoe 🇺🇸 United States and 3 moreCrusoe AI cloud and modular data centre operator that also builds its own power supplyNeocloud · Nvidia Blackwell Radiant 🇬🇧 United KingdomRadiant AI compute infrastructure company building repeatable 'AI factories' powered by its FlightDeck platform.Neocloud · Nvidia Blackwell Verda 🇫🇮 FinlandVerda European full-stack AI cloud offering GPU clusters, instances and model APIsNeocloud · Nvidia Blackwell Vast.ai 🇺🇸 United StatesVast.ai Marketplace for distributed GPU cloud computeMarketplace · Nvidia Blackwell · 2016 GPU Mart 🇺🇸 United StatesGPU Mart Dedicated GPU VPS and bare-metal servers for AI and rendering.Neocloud · Nvidia Blackwell · 2021 Hyperbolic 🇺🇸 United StatesHyperbolic Unified platform for global GPU compute access and orchestrationMarketplace · Nvidia Hopper RunPod 🇺🇸 United StatesRunPod AI developer cloud for training and inferenceNeocloud · Nvidia Blackwell Lambda 🇺🇸 United StatesLambda AI-only GPU cloud running NVIDIA supercomputers for training and inferenceNeocloud · Nvidia Blackwell · 2012 Phala Cloud 🇺🇸 United StatesPhala Cloud Verifiable private AI compute on GPU TEEsNeocloud · Nvidia Hopper DigiPowerX 🇺🇸 United StatesDigiPowerX Vertically integrated AI infrastructure with owned power and GPU compute.Neocloud · Nvidia Blackwell Avian 🇺🇸 United StatesAvian Serverless and dedicated GPU inference for open-source modelsInference platform · Nvidia Blackwell E2E Networks 🇮🇳 IndiaE2E Networks India's listed GPU cloud for AI training and inference.Neocloud · Nvidia Hopper · 2009

Photo credits on this page

The small pictures above come from Wikimedia Commons under free licences. Some are cropped to fit.

The weekly digest

Every Monday: the provider of the day, new pages on EveryNeocloud, and the conferences and anniversaries of the week ahead.