Inference platforms
Fireworks, Baseten, Replicate, Modal and the clouds that serve models by the token. 34 providers on EveryNeocloud so far.
An inference platform runs trained AI models for customers and charges for the work, usually by the token, the small chunk of text a language model reads or writes. Customers send a request to an API and get an answer back, without renting a whole GPU. Fireworks, Baseten, Together AI, Replicate and Modal are known examples, along with the chip companies Groq, Cerebras and SambaNova.
Training versus inference
Training a model is a long job that keeps thousands of GPUs busy for weeks. Inference is the opposite: millions of short requests that must be answered in a fraction of a second. Inference platforms pack many customers onto shared GPUs, batch requests together, cache repeated text and use tricks such as quantisation, storing numbers with fewer bits, to cut cost and delay. As AI tools spread, inference has become a larger share of all AI computing.
How they sell
Most publish prices per million input and output tokens for popular open models such as Llama, Qwen, DeepSeek and Mistral, and offer dedicated deployments for larger customers. Many rent their GPUs from neoclouds rather than owning them. The hyperscalers compete with services such as Amazon Bedrock, and routers such as OpenRouter send each request to whichever provider is cheapest or fastest.
Famous inference platform providers
Still to be written: Hugging Face.
Further reading
Photo credits on this page
The small pictures above come from Wikimedia Commons under free licences. Some are cropped to fit.
- Cerebras: Coolcaesar, CC BY 4.0, source
- Cloudflare: HaeB, CC BY-SA 4.0, source
- Groq: Avivweinstein, CC BY-SA 4.0, source
- SpaceX Super Intelligence: BrokenSegue, CC BY-SA 4.0, source
- OpenRouter: Wikideas1, CC0, source