公开(公告)号：US20230087642A1

公开(公告)日：2023-03-23

申请号：US17991683

申请日：2022-11-21

Applicant: HUAWEI TECHNOLOGIES CO., LTD.

Inventor： Chao Chen ,  Bin Xu ,  Weiping Huang

IPC: G06N3/08

Abstract: A training apparatus used for model training on a neural network in the field of artificial intelligence (AI) is disclosed. The training apparatus includes a plurality of accelerators. In a parallel processing process in which the training apparatus trains a neural network model using the plurality of accelerators, a complete weight coefficient of the neural network model is stored in the plurality of accelerators in the training apparatus in a distributed manner. Weight coefficients of the plurality of accelerators are subsequently added to obtain the complete weight coefficient. The neural network model is further trained on each accelerator based on different input data and the complete weight coefficient. Thus, the complete weight coefficient is stored in the plurality of accelerators in the training apparatus in a distributed manner, to reduce video random access memory (RAM) consumption of the training apparatus in a training process of the neural network model.