公开(公告)号：US11630994B2

公开(公告)日：2023-04-18

申请号：US15898433

申请日：2018-02-17

Applicant: Advanced Micro Devices, Inc.

Inventor： Khaled Hamidouche ,  Michael W LeBeane ,  Walter B Benton ,  Michael L Chu

IPC: G06N3/08 ,  G06F15/173 ,  G06N3/084 ,  G06N3/063 ,  G06N3/045

Abstract: A method of training a neural network includes, at a local computing node, receiving remote parameters from a set of one or more remote computing nodes, initiating execution of a forward pass in a local neural network in the local computing node to determine a final output based on the remote parameters, initiating execution of a backward pass in the local neural network to determine updated parameters for the local neural network, and prior to completion of the backward pass, transmitting a subset of the updated parameters to the set of remote computing nodes.

公开(公告)号：US20190258924A1

公开(公告)日：2019-08-22

申请号：US15898433

申请日：2018-02-17

Applicant: Advanced Micro Devices, Inc.

Inventor： Khaled Hamidouche ,  Michael W LeBeane ,  Walter B Benton ,  Michael L Chu

IPC: G06N3/08 ,  G06F15/173

Abstract: A method of training a neural network includes, at a local computing node, receiving remote parameters from a set of one or more remote computing nodes, initiating execution of a forward pass in a local neural network in the local computing node to determine a final output based on the remote parameters, initiating execution of a backward pass in the local neural network to determine updated parameters for the local neural network, and prior to completion of the backward pass, transmitting a subset of the updated parameters to the set of remote computing nodes.