Tokens per second
How fast a model produces text, measured for one user or for a whole server, the key speed figure for inference.
Also: TPS, output speed, throughput
Tokens per second is quoted two ways. Per-user speed is how quickly one reply streams out; 50 to 100 tokens a second reads faster than people do, and specialised chips from Groq and Cerebras advertise many hundreds or more. System throughput is the total across all users a server handles at once, which decides cost per token. The two pull against each other: serving more users in a batch raises throughput but slows each reply. Benchmarks should state which is meant, the model, and the length of input and output.