公开(公告)号：US20230066545A1

公开(公告)日：2023-03-02

申请号：US17964563

申请日：2022-10-12

Applicant: Google LLC

Inventor： Hugh Brendan McMahan ,  Kunal Talwar ,  Li Zhang ,  Daniel Ramage

IPC: G06N20/00 ,  G06K9/62 ,  G06F8/65 ,  H04L67/10 ,  G06F21/62 ,  H04L9/40

Abstract: Systems and methods for learning differentially private machine-learned models are provided. A computing system can include one or more server computing devices comprising one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors cause the one or more server computing devices to perform operations. The operations can include selecting a subset of client computing devices from a pool of available client computing devices; providing a machine-learned model to the selected client computing devices; receiving, from each selected client computing device, a local update for the machine-learned model; determining a differentially private aggregate of the local updates; and determining an updated machine-learned model based at least in part on the data-weighted average of the local updates.

公开(公告)号：US11475350B2

公开(公告)日：2022-10-18

申请号：US15877196

申请日：2018-01-22

Applicant: Google LLC

Inventor： Hugh Brendan McMahan ,  Kunal Talwar ,  Li Zhang ,  Daniel Ramage

IPC: G06F9/44 ,  G06N20/00 ,  G06K9/62 ,  G06F8/65 ,  H04L67/10 ,  G06F21/62 ,  H04L9/40

Abstract: Systems and methods for learning differentially private machine-learned models are provided. A computing system can include one or more server computing devices comprising one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors cause the one or more server computing devices to perform operations. The operations can include selecting a subset of client computing devices from a pool of available client computing devices; providing a machine-learned model to the selected client computing devices; receiving, from each selected client computing device, a local update for the machine-learned model; determining a differentially private aggregate of the local updates; and determining an updated machine-learned model based at least in part on the data-weighted average of the local updates.

公开(公告)号：US20210158211A1

公开(公告)日：2021-05-27

申请号：US16953977

申请日：2020-11-20

Applicant: Google LLC

Inventor： Kunal Talwar ,  Vitaly Feldman ,  Tomer Koren

IPC: G06N20/00 ,  G06F17/18 ,  G06F21/60

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for training a machine learning model. The method includes obtaining a training data set comprising a plurality of training examples; determining i) a stochastic gradient descent step size schedule, ii) a stochastic gradient descent noise schedule, and iii) a stochastic gradient descent batch size schedule, wherein the stochastic gradient descent batch size schedule comprises a sequence of varying batch sizes; and training a machine learning model on the training data set, comprising performing stochastic gradient descent according to the i) stochastic gradient descent step size schedule, ii) stochastic gradient descent noise schedule, and iii) stochastic gradient descent batch size schedule to adjust a machine learning model loss function.

公开(公告)号：US11726769B2

公开(公告)日：2023-08-15

申请号：US17964563

申请日：2022-10-12

Applicant: Google LLC

Inventor： Hugh Brendan McMahan ,  Kunal Talwar ,  Li Zhang ,  Daniel Ramage

IPC: G06F9/44 ,  G06F8/65 ,  G06N20/00 ,  H04L67/10 ,  G06F21/62 ,  H04L9/40 ,  G06F18/214 ,  G06V10/82 ,  G06V10/94

CPC classification number: G06F8/65 ,  G06F18/214 ,  G06F21/6245 ,  G06N20/00 ,  H04L63/00 ,  H04L67/10 ,  G06V10/82 ,  G06V10/95

Abstract: Systems and methods for learning differentially private machine-learned models are provided. A computing system can include one or more server computing devices comprising one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors cause the one or more server computing devices to perform operations. The operations can include selecting a subset of client computing devices from a pool of available client computing devices; providing a machine-learned model to the selected client computing devices; receiving, from each selected client computing device, a local update for the machine-learned model; determining a differentially private aggregate of the local updates; and determining an updated machine-learned model based at least in part on the data-weighted average of the local updates.

公开(公告)号：US20190227980A1

公开(公告)日：2019-07-25

申请号：US15877196

申请日：2018-01-22

Applicant: Google LLC

Inventor： Hugh Brendan McMahan ,  Kunal Talwar ,  Li Zhang ,  Daniel Ramage

IPC: G06F15/18 ,  G06F8/65 ,  G06K9/62

Abstract: Systems and methods for learning differentially private machine-learned models are provided. A computing system can include one or more server computing devices comprising one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors cause the one or more server computing devices to perform operations. The operations can include selecting a subset of client computing devices from a pool of available client computing devices; providing a machine-learned model to the selected client computing devices; receiving, from each selected client computing device, a local update for the machine-learned model; determining a differentially private aggregate of the local updates; and determining an updated machine-learned model based at least in part on the data-weighted average of the local updates.