公开(公告)号：US20250078812A1

公开(公告)日：2025-03-06

申请号：US18794773

申请日：2024-08-05

Applicant: GOOGLE LLC

Inventor： Yonghui Xiao ,  Françoise Beaufays ,  Yuxin Ding

IPC: G10L15/06 ,  G06N3/098 ,  G10L15/183 ,  G10L15/30

Abstract: Implementations described herein are directed to a framework for decentralized learning of large global machine learning (ML) model(s). In various implementations, remote processor(s) of a remote system can identify a global ML model, select client devices to participate in a given round of decentralized learning of the global ML model, and transmit, to each of the client devices, a processed version of the global ML model that is of a reduced transferrable size. Further, client device processor(s) of a client device can receive the processed version of the global ML model, obtain corresponding client data, perform partial model training, based on processing the corresponding client data, for the processed version of the global ML model to generate a corresponding update, and transmit the corresponding update back to the remote system. Moreover, the remote processor(s) can update, based on at least the corresponding update, the global ML model.

公开(公告)号：US20240371362A1

公开(公告)日：2024-11-07

申请号：US18652587

申请日：2024-05-01

Applicant: GOOGLE LLC

Inventor： Tien-Ju Yang ,  Yonghui Xiao ,  Giovanni Motta ,  Françoise Beaufays ,  Rajiv Mathews ,  Mingqing Chen

IPC: G10L15/06

Abstract: Implementations are directed to efficient federated learning of machine learning (ML) model(s) through on-the-fly decompression and compression of model parameters, of the ML model(s), when facilitating forward propagation and/or back propagation at client device(s). For example, implementations can transmit, from a remote system to a client device, a compressed on-device ML model that includes some compressed parameters. Further, the client device can, in performing forward propagation and/or back propagation using the on-device ML model, decompress those compressed parameters on-the-fly as the parameters are needed for the propagation. The propagation will utilize the decompressed parameters that were decompressed on the fly. Further, after the decompressed parameters are utilized, they can be deallocated from memory (while their compressed counterparts optionally remain in memory) to enable allocation of memory for further decompressed parameters that will be needed next and/or needed for other ongoing process(es).