Mixture of experts
A model design where each token is sent to only a few of many smaller sub-networks, so a huge model runs at the cost of a smaller one.
Also: MoE
In a mixture-of-experts model, some layers are split into many "experts", and a small router picks a few for each token. DeepSeek-V3, released in December 2024, has 671 billion parameters in total but uses about 37 billion per token. This cuts compute per token, but all experts must still sit in memory, and tokens travel between GPUs to reach their experts, which places heavy demands on the network. Large NVLink domains such as the NVL72 suit these models.