LPU
Groq's chip for running language models, which keeps model weights in fast on-chip memory to generate tokens quickly.
Also: language processing unit
The language processing unit is the accelerator designed by Groq, a company founded in 2016 by engineers who had worked on Google's TPU. Instead of the stacked HBM memory used by GPUs, each LPU uses on-chip SRAM, which is much faster but much smaller, so a large model is spread across hundreds of chips. Its compiler plans every step ahead of time, which gives predictable speed. Groq sells access mainly through its own cloud, GroqCloud, priced by the token, and has built data centres in the United States and Saudi Arabia.