Scaling laws
Observed rules showing that models get predictably better as compute, data and size grow, the reason labs build ever larger clusters.
Also: Chinchilla scaling
Research in 2020 by OpenAI, and in 2022 by DeepMind in the "Chinchilla" paper, showed that a language model's error falls smoothly and predictably as training compute, parameters and data increase. Chinchilla suggested about 20 training tokens per parameter for the best use of a fixed budget, though labs now often train smaller models on far more data so they are cheaper to serve. These laws underpin the demand for compute, and debate over whether they are slowing is followed closely by the whole industry.