公开(公告)号：US20240256966A1

公开(公告)日：2024-08-01

申请号：US18424660

申请日：2024-01-26

Applicant: Google LLC

Inventor： Ankush Garg ,  Yichi Zhang ,  Yuan Cao ,  Lukasz Lew ,  Orhan Firat ,  Behrooz Ghorbani

IPC: G06N20/00

CPC classification number: G06N20/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing sequence generation tasks using binarized neural networks. The binarized neural network is an attention neural network configured to perform the task and the attention neural network includes a plurality of attention blocks, with each block including an attention block and a binarized feedforward block.

公开(公告)号：US20230214642A1

公开(公告)日：2023-07-06

申请号：US17568933

申请日：2022-01-05

Applicant: Google LLC

Inventor： Hakim Sidahmed ,  Zheng Xu ,  Mingqing Chen ,  Yuan Cao ,  Ankush Garg

IPC: G06N3/08

CPC classification number: G06N3/08

Abstract: Example aspects of the present disclosure provide a novel, resource-efficient approach for federated machine learning techniques with PTNs. The system can determine a first set of training parameters from a plurality of parameters of the global model. Additionally, the system can generate a random seed, using a random number generator, based on a set of frozen parameters. Moreover, the system can transmit, respectively to a plurality of client computing devices, a first set of training parameters and the random seed. Furthermore, the system can receive, respectively from the plurality of client computing devices, updates to one or more parameters in the first set of training parameters. Subsequently, the system can aggregate the updates to one or more parameters that are respectively received from the plurality of client computing devices. The system can modify one or more global parameters of the global model based on the aggregation.