公开(公告)号：US20250077871A1

公开(公告)日：2025-03-06

申请号：US18564160

申请日：2023-05-25

Applicant: Google LLC

Inventor： Devora Berlowitz ,  Steve Shaw-Tang Chien ,  Yunqi Xue ,  Lin Ning ,  Shuang Song ,  Mei Chen

IPC: G06N3/084

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for privacy-sensitive training of a neural network. In one aspect, a system comprises a central memory configured to store current values of a set of neural network parameters and one or more computers that are configured to implement a plurality of worker computing units, where each worker computing unit is configured to repeatedly perform operations comprising obtaining current values of the set of neural network parameters from the central memory, sampling a batch of network inputs from a set of training data, determining a respective gradient corresponding to each network input, determining an aggregated gradient based on the gradients, identifying a subset of a set of gradient values as target values, generating a noisy gradient by combining random noise with the target gradient values, and updating the current values of the set of neural network parameters.

公开(公告)号：US20240054391A1

公开(公告)日：2024-02-15

申请号：US17928372

申请日：2022-04-05

Applicant: GOOGLE LLC

Inventor： Abhradeep Guha Thakurta ,  Li Zhang ,  Prateek Jain ,  Shuang Song ,  Steffen Rendle ,  Steve Shaw-Tang Chien ,  Walid Krichene ,  Yarong Mu

IPC: G06N20/00 ,  G06F21/62

CPC classification number: G06N20/00 ,  G06F21/6218

Abstract: Computer-implemented systems and methods for training a decentralized model for making a personalized recommendation. In one aspect, the method comprising: obtaining, using user activity data, client-side training data that includes features and training labels; and training, by the client device, a decentralized model in training rounds, wherein training, in each training round comprises: receiving, first data including a current server-side embedding generated by the server-side machine learning model, wherein the first data received from the server does not include any server-side data used in generating the current server-side embedding; generating, using the client-side machine learning model, a client-side embedding based on the client-side training data; updating, using the client-side embedding and the current server-side embedding and based on the training labels, the client-side machine learning model; generating, an updated client-side embedding; and transmitting second data including the updated client-side embedding for subsequent updating of the server-side machine learning model.