Serverless GPU
GPUs that start when a request arrives and shut down when it ends, billed by the second, so idle time costs nothing.
Also: pay-per-second GPUs
On a serverless GPU platform, a developer uploads code or a container and the platform runs it on a GPU only when called, scaling from zero to many copies and back. Billing is by the second of use. Modal, RunPod, Replicate, Baseten and the hyperscalers offer versions of this. It suits uneven traffic, prototypes and batch jobs. The trade-off is the cold start: loading a large model onto a fresh GPU can take seconds to minutes, which platforms reduce with caching and pre-warmed machines.