公开(公告)号：US20250103959A1

公开(公告)日：2025-03-27

申请号：US18885339

申请日：2024-09-13

Applicant: Beijing Baidu Netcom Science Technology Co., Ltd.

Inventor： Liang Shen ,  Dianhai Yu ,  Weibao Gong ,  Jinle Zeng ,  Haifeng Wang

IPC: G06N20/00

Abstract: Provided is a performance optimization method for a model training device, an electronic device, and a storage medium, relating to the fields of deep learning, large model training, and distributed parallel strategies. The method includes: determining communication timing of a current model training device with respect to a target model block at a target sorting position, so as to be able to perform synchronously collective communication with other model training devices of a plurality of model training devices with respect to model blocks at the target sorting position; and performing the collective communication on a backward gradient of the target model block at the communication timing.