BF16
A 16-bit number format created at Google Brain that keeps the range of 32-bit numbers with less detail, now standard for AI training.
Also: bfloat16, brain floating point
Bfloat16 uses 16 bits but keeps the same 8-bit exponent as the 32-bit format, so it can represent very large and very small numbers, while storing fewer digits of detail. Neural network training turns out to care more about range than detail, so BF16 trains stably without the tricks needed for the older FP16 format. Google introduced it on its TPUs, and Nvidia, AMD and Intel adopted it. Most large models of the 2020s were trained mainly in BF16.