Test-time compute
Letting a model spend more computation while answering, for example by reasoning step by step, to get better results.
Also: reasoning models, inference-time scaling
Reasoning models, such as OpenAI's o-series from 2024 and DeepSeek-R1 from January 2025, generate long internal chains of thought before giving an answer. The more tokens they spend thinking, the better they tend to do on hard problems. This shifts compute from training to inference: a single hard question can use thousands of tokens. For providers it means demand for inference capacity grows faster than the number of users, and it favours hardware that generates tokens quickly.