公开(公告)号：US20230223028A1

公开(公告)日：2023-07-13

申请号：US18007656

申请日：2020-10-16

Applicant: GOOGLE LLC

Inventor： Om Thakkar ,  Abhradeep Guha Thakurta ,  Peter Kairouz ,  Borja de Balle Pigem ,  Brendan McMahan

IPC: G10L15/30 ,  G10L15/06

CPC classification number: G10L15/30 ,  G10L15/063

Abstract: Techniques are disclosed that enable training a global model using gradients provided to a remote system by a set of client devices during a reporting window, where each client device randomly determines a reporting time in the reporting window to provide the gradient to the remote system. Various implementations include each client device determining a corresponding gradient by processing data using a local model stored locally at the client device, where the local model corresponds to the global model.

公开(公告)号：US20240054391A1

公开(公告)日：2024-02-15

申请号：US17928372

申请日：2022-04-05

Applicant: GOOGLE LLC

Inventor： Abhradeep Guha Thakurta ,  Li Zhang ,  Prateek Jain ,  Shuang Song ,  Steffen Rendle ,  Steve Shaw-Tang Chien ,  Walid Krichene ,  Yarong Mu

IPC: G06N20/00 ,  G06F21/62

CPC classification number: G06N20/00 ,  G06F21/6218

Abstract: Computer-implemented systems and methods for training a decentralized model for making a personalized recommendation. In one aspect, the method comprising: obtaining, using user activity data, client-side training data that includes features and training labels; and training, by the client device, a decentralized model in training rounds, wherein training, in each training round comprises: receiving, first data including a current server-side embedding generated by the server-side machine learning model, wherein the first data received from the server does not include any server-side data used in generating the current server-side embedding; generating, using the client-side machine learning model, a client-side embedding based on the client-side training data; updating, using the client-side embedding and the current server-side embedding and based on the training labels, the client-side machine learning model; generating, an updated client-side embedding; and transmitting second data including the updated client-side embedding for subsequent updating of the server-side machine learning model.

公开(公告)号：US20240095594A1

公开(公告)日：2024-03-21

申请号：US18459354

申请日：2023-08-31

Applicant: Google LLC

Inventor： Om Dipakbhai Thakkar ,  Arun Ganesh ,  Virat Vishnu Shejwalkar ,  Abhradeep Guha Thakurta ,  Rajiv Mathews

IPC: G06N20/00

CPC classification number: G06N20/00

Abstract: A method includes training a first differentially private (DP) model using a private training set, the private training set including a plurality of training samples, the first DP model satisfying a differential privacy budget, the differential privacy budget defining an amount of information about individual training samples of the private training set that may be revealed by the first DP model. The method also includes, while training the first DP model, generating a plurality of intermediate checkpoints, each intermediate checkpoint of the plurality of intermediate checkpoints representing a different intermediate state of the first DP model, each of the intermediate checkpoints satisfying the same differential privacy budget. The method further includes determining an aggregate of the first DP model and the plurality of intermediate checkpoints, and determining, using the aggregate, a second DP model, the second DP model satisfying the same differential privacy budget.