In order to operate, AI chips require both traditional dynamic RAM (DRAM) and high bandwidth memory (HBM). Dynamic RAM is used for the operating system, applications and orchestration. High bandwidth memory (HBM) is used to hold the neuron weights in the neural network, where all the computation is done during training and inference. See
neural network,
dynamic RAM and
high bandwidth memory.