News
DealsCorvex agrees USD 103 million take-or-pay deal to perform AI inference with Mid-Atlantic GPUsSovereignNeysa scales AI infrastructure in India with AMD Instinct MI350X GPUs on TytanInferenceMicrosoft Foundry adds Baseten, DeepSeek AI inference (MSFT:NASDAQ)ClustersDataOne and Rowan University sign engineering co-op program agreement for DataOne's Vineland AI factoryClustersLook inside Cal Poly's $3 million AI Factory (Video)DealsOpenAI Taps AMD Chips for 457 MW of AI Capacity in Six-Year TensorWave DealClustersCal Poly launches an AI data center on campus. Take a look insideInferenceArista takes aim at the next AI network battleground: distributed inferenceDealsOpenAI Signs 457 MW AI Data Center Deal as AMD Expands Its Challenge to NvidiaClustersBehind Dell AI Factory’s NVIDIA & AMD Enterprise ExpansionDealsCorvex agrees USD 103 million take-or-pay deal to perform AI inference with Mid-Atlantic GPUsSovereignNeysa scales AI infrastructure in India with AMD Instinct MI350X GPUs on TytanInferenceMicrosoft Foundry adds Baseten, DeepSeek AI inference (MSFT:NASDAQ)ClustersDataOne and Rowan University sign engineering co-op program agreement for DataOne's Vineland AI factoryClustersLook inside Cal Poly's $3 million AI Factory (Video)DealsOpenAI Taps AMD Chips for 457 MW of AI Capacity in Six-Year TensorWave DealClustersCal Poly launches an AI data center on campus. Take a look insideInferenceArista takes aim at the next AI network battleground: distributed inferenceDealsOpenAI Signs 457 MW AI Data Center Deal as AMD Expands Its Challenge to NvidiaClustersBehind Dell AI Factory’s NVIDIA & AMD Enterprise Expansion
EveryNeocloud

Tokens per second

How fast a model produces text, measured for one user or for a whole server, the key speed figure for inference.

Also: TPS, output speed, throughput

Tokens per second is quoted two ways. Per-user speed is how quickly one reply streams out; 50 to 100 tokens a second reads faster than people do, and specialised chips from Groq and Cerebras advertise many hundreds or more. System throughput is the total across all users a server handles at once, which decides cost per token. The two pull against each other: serving more users in a batch raises throughput but slows each reply. Benchmarks should state which is meant, the model, and the length of input and output.

See also

Find providers that mention Tokens per second

The weekly digest

Every Monday: the provider of the day, new pages on EveryNeocloud, and the conferences and anniversaries of the week ahead.