Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating control signals for controlling a magnetic field for confining plasma in a chamber of a magnetic confinement device. One of the methods includes, for each of a plurality of time steps, obtaining an observation characterizing a current state of the plasma in the chamber of the magnetic confinement device, processing an input including the observation using a plasma confinement neural network to generate a magnetic control output that characterizes control signals for controlling the magnetic field of the magnetic confinement device, and generating the control signals for controlling the magnetic field of the magnetic confinement device based on the magnetic control output.