A system and a method for training a parameter set in a neural network includes a main-control-node set, used for controlling a training process and storing a data set and a parameter set that are used for training, where the main-control-node set includes M main control nodes, every two of the M main control nodes are in a communication connection, and at least one main control node of the M main control nodes is configured to back up the parameter set. The system also includes N training-node sets, where the training-node set includes multiple training nodes, and the training node is configured to perform training according to a data set and a parameter set that are delivered by the main-control-node set, and send a training result to a corresponding main control node.