Inference
Running a trained model to produce answers, images or other output for users, as opposed to training it.
Also: serving
Inference is the use of a model: a prompt goes in and tokens come out. Each request is small compared with training, but popular services handle billions of requests, so inference is expected to use the larger share of AI compute over time. It has two phases, reading the prompt (prefill) and generating the reply token by token (decode), with different demands on the hardware. Inference cares about speed per user, cost per token and being close to users, which supports a wider range of chips and smaller, more distributed sites than training.