News
DealsCorvex agrees USD 103 million take-or-pay deal to perform AI inference with Mid-Atlantic GPUsSovereignNeysa scales AI infrastructure in India with AMD Instinct MI350X GPUs on TytanInferenceMicrosoft Foundry adds Baseten, DeepSeek AI inference (MSFT:NASDAQ)ClustersDataOne and Rowan University sign engineering co-op program agreement for DataOne's Vineland AI factoryClustersLook inside Cal Poly's $3 million AI Factory (Video)DealsOpenAI Taps AMD Chips for 457 MW of AI Capacity in Six-Year TensorWave DealClustersCal Poly launches an AI data center on campus. Take a look insideInferenceArista takes aim at the next AI network battleground: distributed inferenceDealsOpenAI Signs 457 MW AI Data Center Deal as AMD Expands Its Challenge to NvidiaClustersBehind Dell AI Factory’s NVIDIA & AMD Enterprise ExpansionDealsCorvex agrees USD 103 million take-or-pay deal to perform AI inference with Mid-Atlantic GPUsSovereignNeysa scales AI infrastructure in India with AMD Instinct MI350X GPUs on TytanInferenceMicrosoft Foundry adds Baseten, DeepSeek AI inference (MSFT:NASDAQ)ClustersDataOne and Rowan University sign engineering co-op program agreement for DataOne's Vineland AI factoryClustersLook inside Cal Poly's $3 million AI Factory (Video)DealsOpenAI Taps AMD Chips for 457 MW of AI Capacity in Six-Year TensorWave DealClustersCal Poly launches an AI data center on campus. Take a look insideInferenceArista takes aim at the next AI network battleground: distributed inferenceDealsOpenAI Signs 457 MW AI Data Center Deal as AMD Expands Its Challenge to NvidiaClustersBehind Dell AI Factory’s NVIDIA & AMD Enterprise Expansion
EveryNeocloud

Inference platforms

Fireworks, Baseten, Replicate, Modal and the clouds that serve models by the token. 34 providers on EveryNeocloud so far.

An inference platform runs trained AI models for customers and charges for the work, usually by the token, the small chunk of text a language model reads or writes. Customers send a request to an API and get an answer back, without renting a whole GPU. Fireworks, Baseten, Together AI, Replicate and Modal are known examples, along with the chip companies Groq, Cerebras and SambaNova.

Training versus inference

Training a model is a long job that keeps thousands of GPUs busy for weeks. Inference is the opposite: millions of short requests that must be answered in a fraction of a second. Inference platforms pack many customers onto shared GPUs, batch requests together, cache repeated text and use tricks such as quantisation, storing numbers with fewer bits, to cut cost and delay. As AI tools spread, inference has become a larger share of all AI computing.

How they sell

Most publish prices per million input and output tokens for popular open models such as Llama, Qwen, DeepSeek and Mistral, and offer dedicated deployments for larger customers. Many rent their GPUs from neoclouds rather than owning them. The hyperscalers compete with services such as Amazon Bedrock, and routers such as OpenRouter send each request to whichever provider is cheapest or fastest.

Famous inference platform providers

Still to be written: Hugging Face.

Further reading

Amazon Bedrock 🇺🇸 United StatesAmazon Bedrock Serverless platform giving one API to many foundation modelsInference platform · 2023 🇺🇸 United StatesSpaceX Super Intelligence Musk's AI company behind Grok and the Colossus data centres, now part of SpaceX.Neocloud · Nvidia Hopper · 2023 🇺🇸 United States and 2 moreGroq Chip designer and inference cloud built on its own Language Processing UnitChipmaker cloud · Groq LPU · 2016 Gcore 🇱🇺 Luxembourg and 1 moreGcore Luxembourg edge AI, cloud, CDN and security provider with GPU clusters in EuropeNeocloud · Other Nvidia GPUs · 2011 🇺🇸 United StatesOpenRouter One API that routes requests to hundreds of AI modelsInference platform · 2023 🇺🇸 United StatesCloudflare Edge network running Nvidia GPUs for AI inference via Workers AIInference platform · Other Nvidia GPUs · 2009 Jarvislabs 🇮🇳 IndiaJarvislabs GPU cloud and inference platform based in IndiaNeocloud · Nvidia Hopper Chutes WorldChutes Serverless, decentralised compute for running open-source AI models in productionInference platform Together AI 🇺🇸 United StatesTogether AI AI cloud running open models on demand, with GPU clusters and inference researchNeocloud · Nvidia Blackwell Crusoe 🇺🇸 United States and 3 moreCrusoe AI cloud and modular data centre operator that also builds its own power supplyNeocloud · Nvidia Blackwell Verda 🇫🇮 FinlandVerda European full-stack AI cloud offering GPU clusters, instances and model APIsNeocloud · Nvidia Blackwell Fal 🇺🇸 United StatesFal Serverless GPU platform for generative AI inferenceInference platform · Other Nvidia GPUs · 2021 Novita AI 🇺🇸 United StatesNovita AI AI cloud platform offering model APIs and GPU instances for developers.Neocloud · Other Nvidia GPUs Modal 🇺🇸 United States and 1 moreModal Serverless cloud platform for AI development and inferenceDeveloper cloud · Other Nvidia GPUs DeepInfra 🇺🇸 United StatesDeepInfra Inference cloud for open-source AI modelsInference platform · Nvidia Blackwell · 2022 Featherless AI 🇺🇸 United StatesFeatherless AI Serverless inference platform for open-source AI modelsInference platform · AMD Instinct · 2023 Cerebrium 🇿🇦 South AfricaCerebrium Serverless GPU infrastructure for real-time AI workloadsInference platform · Nvidia Hopper Inference.net 🇺🇸 United StatesInference.net Marketplace for low-cost AI inference via unused GPU capacityInference platform PPIO 🇨🇳 ChinaPPIO Independent AI cloud for agent infrastructure and GPU computeNeocloud · Other Nvidia GPUs · 2018 Fireworks AI 🇺🇸 United StatesFireworks AI Platform for running and training open AI modelsInference platform · Other Nvidia GPUs Runware 🇺🇸 United StatesRunware Serverless AI inference platform with custom hardwareInference platform · Other Nvidia GPUs · 2023 Saturn Cloud 🇺🇸 United StatesSaturn Cloud Managed inference platform for open AI models on dedicated GPUsInference platform · Other Nvidia GPUs Zenlayer 🇺🇸 United StatesZenlayer Distributed cloud infrastructure for global AI inferenceNeocloud · 2014 Livepeer WorldLivepeer Open network for AI video inference workloadsMarketplace · Other Nvidia GPUs Inferless 🇺🇸 United StatesInferless Serverless GPU inference platform for custom modelsInference platform · Other Nvidia GPUs Avian 🇺🇸 United StatesAvian Serverless and dedicated GPU inference for open-source modelsInference platform · Nvidia Blackwell FriendliAI 🇰🇷 South KoreaFriendliAI Accelerated generative AI inference serving platformInference platform · Other Nvidia GPUs Replicate WorldReplicate Run, fine-tune and deploy AI models through one APIInference platform Simplismart 🇮🇳 IndiaSimplismart AI inference platform for custom model deployment and scalingInference platform · Nvidia Blackwell GMI Cloud WorldGMI Cloud AI-native cloud for production inference, GPU clusters and agent runtimesNeocloud · Nvidia Hopper Hot Aisle WorldHot Aisle Developer-first cloud for sovereign inference on isolated AMD GPUsNeocloud · AMD Instinct · 2023 Baseten 🇺🇸 United StatesBaseten Inference cloud for running AI models in productionInference platform Parasail 🇺🇸 United StatesParasail Inference cloud for AI startups, billed by token.Inference platform · Other and custom chips Neysa 🇮🇳 IndiaNeysa GPU cloud and AI platform for enterprises in IndiaNeocloud · Other Nvidia GPUs

Photo credits on this page

The small pictures above come from Wikimedia Commons under free licences. Some are cropped to fit.

The weekly digest

Every Monday: the provider of the day, new pages on EveryNeocloud, and the conferences and anniversaries of the week ahead.