Sparsity
Skipping calculations on values that are zero; chip makers quote speeds with sparsity that real models often cannot reach.
Also: structured sparsity
Many values inside a neural network are zero or close to it. If the hardware can skip them, it does less work. Since the A100 in 2020, Nvidia GPUs support "2:4 structured sparsity", where two of every four values are zero, doubling peak speed. Spec sheets often quote this doubled figure, but most models are not trained to fit that pattern, so the dense figure, half as high, is the safer comparison. Mixture-of-experts models are a different, coarser kind of sparsity.