Time to first token
How long a user waits between sending a prompt and seeing the first word of the reply.
Also: TTFT
Time to first token measures the delay before output begins. It covers the network trip, any queueing, and the prefill phase in which the model reads the whole prompt. Long prompts, such as a large document, raise it, and so do busy servers. For chat a TTFT under a second feels responsive; voice assistants and coding tools want less. Providers reduce it with faster chips, by separating prefill from decode onto different GPUs, and by caching prompts that repeat.