公开(公告)号：US20190385063A1

公开(公告)日：2019-12-19

申请号：US16442203

申请日：2019-06-14

Applicant: GOOGLE LLC

Inventor： Xinnan Yu ,  Shanshan Wu ,  Daniel Holtmann-Rice ,  Dmitry Storcheus ,  Sanjiv Kumar ,  Afshin Rostamizadeh

IPC: G06N3/08 ,  G06N3/04 ,  G06F17/16

Abstract: A sparse dataset is encoded using a data-driven learned sensing matrix. For example, an example method includes receiving a dataset of sparse vectors with dimension d from a requesting process, initializing an encoding matrix of dimension k×d, selecting a subset of sparse vectors from the dataset, and updating the encoding matrix via machine learning. Updating the encoding matrix includes using a linear encoder to generate an encoded vector of dimension k for each vector in the subset, the linear encoder using the encoding matrix, using a non-linear decoder to decode each of the encoded vectors, the non-linear decoder using a transpose of the encoding matrix in a projected subgradient, and adjusting the encoding matrix using back propagation. The method also includes returning an embedding of each sparse vector in the dataset of sparse vectors, the embedding being generated with the updated encoding matrix.

公开(公告)号：US12033080B2

公开(公告)日：2024-07-09

申请号：US16442203

申请日：2019-06-14

Applicant: GOOGLE LLC

Inventor： Xinnan Yu ,  Shanshan Wu ,  Daniel Holtmann-Rice ,  Dmitry Storcheus ,  Sanjiv Kumar ,  Afshin Rostamizadeh

IPC: G06N3/084 ,  G06F17/16 ,  G06N3/02 ,  G06N3/045

CPC classification number: G06N3/084 ,  G06F17/16 ,  G06N3/02 ,  G06N3/045

Abstract: A sparse dataset is encoded using a data-driven learned sensing matrix. For example, an example method includes receiving a dataset of sparse vectors with dimension d from a requesting process, initializing an encoding matrix of dimension k×d, selecting a subset of sparse vectors from the dataset, and updating the encoding matrix via machine learning. Updating the encoding matrix includes using a linear encoder to generate an encoded vector of dimension k for each vector in the subset, the linear encoder using the encoding matrix, using a non-linear decoder to decode each of the encoded vectors, the non-linear decoder using a transpose of the encoding matrix in a projected subgradient, and adjusting the encoding matrix using back propagation. The method also includes returning an embedding of each sparse vector in the dataset of sparse vectors, the embedding being generated with the updated encoding matrix.

公开(公告)号：US20220398500A1

公开(公告)日：2022-12-15

申请号：US17332893

申请日：2021-05-27

Applicant: Google LLC

Inventor： Karan Singhal ,  Hakim Sidahmed, JR. ,  Zachary A. Garrett ,  Shanshan Wu ,  John Keith Rush ,  Sushant Prakash

IPC: G06N20/20

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a machine learning model having a set of local model parameters and a set of global model parameters under a partially local federated learning framework. One of the methods include maintaining local data and data defining the local model parameters; receiving data defining current values of the global model parameters; determining, based on the local data, the local model parameters, and the current values of the global model parameters, current values of the local model parameters; determining, based on the local data, the current values of the local model parameters, and the current values of the global model parameters, updated values of the global model parameters; generating, based on the updated values of the global model parameters, parameter update data defining an update to the global model parameters; and transmitting the parameter update data.