RDMA
A way for one computer to read or write another's memory directly over the network, without involving its processor.
Also: remote direct memory access
Remote direct memory access lets the network card move data straight between the memory of two machines, skipping the operating system and the CPU. This removes copying and delay, which is essential when thousands of GPUs exchange results many times a second during training. InfiniBand was built around RDMA, and RoCE brings it to Ethernet. GPUDirect RDMA goes one step further by letting the network card read and write GPU memory directly.