-
公开(公告)号：US12205005B2
公开(公告)日：2025-01-21
申请号：US16931862
申请日：2020-07-17
Applicant: Google LLC
Inventor： Xinnan Yu , Ankit Singh Rawat , Jiecao Chen , Ananda Theertha Suresh , Sanjiv Kumar
IPC: G06N3/08 , G06F17/14 , G06F17/18 , G06F18/2431 , G06F40/20 , G06N3/084 , G06N20/00 , G06N20/10 , G06V10/77
Abstract: Systems and methods for low bias negative sampling of classes according to the sampled softmax method are described herein. The systems and methods can include training a machine-learned model for classifying inputs into one or more classes of a plurality of classes, each of the plurality of classes having an associated class embedding in a plurality of class embeddings. The systems and methods can include selecting, by the one or more computing devices, one or more negative classes from the plurality of classes based at least in part on a probability distribution approximating a softmax distribution, wherein the probability distribution is determined based at least in part on a Random Fourier Features map.
-
公开(公告)号：US11874866B2
公开(公告)日：2024-01-16
申请号：US18081376
申请日：2022-12-14
Applicant: Google LLC
Inventor： Xiang Wu , David Simcha , Daniel Holtmann-Rice , Sanjiv Kumar , Ananda Theertha Suresh , Ruiqi Guo , Xinnan Yu
CPC classification number: G06F16/3347 , G06F16/313 , G06F16/319 , G06N20/00
Abstract: The present disclosure provides systems and methods that include or otherwise leverage use of a multiscale quantization model that is configured to provide a quantized dataset. In particular, the multiscale quantization model can receive and perform vector quantization of a first dataset. The multiscale quantization model can generate a residual dataset based at least in part on a result of the vector quantization. The multiscale quantization model can apply a rotation matrix to the residual dataset to generate a rotated residual dataset that includes a plurality of rotated residuals. The multiscale quantization model can perform reparameterization of each rotated residual in the rotated residual dataset into a direction component and a scale component. The multiscale quantization model can perform product quantization of the direction components of the plurality of rotated residuals, and perform scalar quantization of the scale components of the plurality of rotated residuals.
-
公开(公告)号：US11531695B2
公开(公告)日：2022-12-20
申请号：US16638802
申请日：2018-05-14
Applicant: Google LLC
Inventor： Xiang Wu , David Simcha , Daniel Holtmann-Rice , Sanjiv Kumar , Ananda Theertha Suresh , Ruiqi Guo , Xinnan Yu
Abstract: The present disclosure provides systems and methods that include or otherwise leverage use of a multiscale quantization model that is configured to provide a quantized dataset. In particular, the multiscale quantization model can receive and perform vector quantization of a first dataset. The multiscale quantization model can generate a residual dataset based at least in part on a result of the vector quantization. The multiscale quantization model can apply a rotation matrix to the residual dataset to generate a rotated residual dataset that includes a plurality of rotated residuals. The multiscale quantization model can perform reparameterization of each rotated residual in the rotated residual dataset into a direction component and a scale component. The multiscale quantization model can perform product quantization of the direction components of the plurality of rotated residuals, and perform scalar quantization of the scale components of the plurality of rotated residuals.
-
公开(公告)号：US20200242514A1
公开(公告)日：2020-07-30
申请号：US16850053
申请日：2020-04-16
Applicant: Google LLC
Inventor： Hugh Brendan McMahan , Dave Morris Bacon , Jakub Konecny , Xinnan Yu
Abstract: The present disclosure provides efficient communication techniques for transmission of model updates within a machine learning framework, such as, for example, a federated learning framework in which a high-quality centralized model is trained on training data distributed overt a large number of clients each with unreliable network connections and low computational power. In an example federated learning setting, in each of a plurality of rounds, each client independently updates the model based on its local data and communicates the updated model back to the server, where all the client-side updates are used to update a global model. The present disclosure provides systems and methods that reduce communication costs. In particular, the present disclosure provides at least: structured update approaches in which the model update is restricted to be small and sketched update approaches in which the model update is compressed before sending to the server.
-
公开(公告)号：US20200183964A1
公开(公告)日：2020-06-11
申请号：US16638802
申请日：2018-05-14
Applicant: Google LLC
Inventor： Xiang Wu , David Simcha , Daniel Holtmann-Rice , Sanjiv Kumar , Ananda Theertha Suresh , Ruiqi Guo , Xinnan Yu
Abstract: The present disclosure provides systems and methods that include or otherwise leverage use of a multiscale quantization model that is configured to provide a quantized dataset. In particular, the multiscale quantization model can receive and perform vector quantization of a first dataset. The multiscale quantization model can generate a residual dataset based at least in part on a result of the vector quantization. The multiscale quantization model can apply a rotation matrix to the residual dataset to generate a rotated residual dataset that includes a plurality of rotated residuals. The multiscale quantization model can perform reparameterization of each rotated residual in the rotated residual dataset into a direction component and a scale component. The multiscale quantization model can perform product quantization of the direction components of the plurality of rotated residuals, and perform scalar quantization of the scale components of the plurality of rotated residuals.
-
Patent Agency Ranking
公开(公告)号：US20240098138A1
公开(公告)日：2024-03-21
申请号：US18240799
申请日：2023-08-31
Applicant: Google LLC
Inventor： Ananda Theertha Suresh , Sanjiv Kumar , Hugh Brendan McMahan , Xinnan Yu
CPC classification number: H04L67/10 , G06F17/12 , G06F17/16 , G06F17/18 , G06N7/01 , G06N20/00 , H03M7/3059 , H03M7/3082 , H03M7/40 , H04L67/01
Abstract: The present disclosure provides systems and methods for communication efficient distributed mean estimation. In particular, aspects of the present disclosure can be implemented by a system in which a number of vectors reside on a number of different clients, and a centralized server device seeks to estimate the mean of such vectors. According to one aspect of the present disclosure, a client computing device can rotate a vector by a random rotation matrix and then subsequently perform probabilistic quantization on the rotated vector. According to another aspect of the present disclosure, subsequent to quantization but prior to transmission, the client computing can encode the quantized vector according to a variable length coding scheme (e.g., by computing variable length codes).
-
公开(公告)号：US20230130021A1
公开(公告)日：2023-04-27
申请号：US17974334
申请日：2022-10-26
Applicant: Google LLC
Inventor： Wittawat Jitkrittum , Michal Mateusz Lukasik , Ananda Theertha Suresh , Xinnan Yu , Gang Wang
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for implementing privacy-preserving machine learning models (e.g., neural networks) in secure multi-part computing environments. Methods can include computing an output of a particular layer of a neural network deployed in a two computing system environment using a cosine activation function.
-
公开(公告)号：US20210019654A1
公开(公告)日：2021-01-21
申请号：US16931862
申请日：2020-07-17
Applicant: Google LLC
Inventor： Xinnan Yu , Ankit Singh Rawat , Jiecao Chen , Ananda Theertha Suresh , Sanjiv Kumar
Abstract: Systems and methods for low bias negative sampling of classes according to the sampled softmax method are described herein. The systems and methods can include training a machine-learned model for classifying inputs into one or more classes of a plurality of classes, each of the plurality of classes having an associated class embedding in a plurality of class embeddings. The systems and methods can include selecting, by the one or more computing devices, one or more negative classes from the plurality of classes based at least in part on a probability distribution approximating a softmax distribution, wherein the probability distribution is determined based at least in part on a Random Fourier Features map.
-
公开(公告)号：US10657461B2
公开(公告)日：2020-05-19
申请号：US16335695
申请日：2017-09-07
Applicant: Google LLC
Inventor： Hugh Brendan McMahan , Dave Morris Bacon , Jakub Konecny , Xinnan Yu
Abstract: The present disclosure provides efficient communication techniques for transmission of model updates within a machine learning framework, such as, for example, a federated learning framework in which a high-quality centralized model is trained on training data distributed overt a large number of clients each with unreliable network connections and low computational power. In an example federated learning setting, in each of a plurality of rounds, each client independently updates the model based on its local data and communicates the updated model back to the server, where all the client-side updates are used to update a global model. The present disclosure provides systems and methods that reduce communication costs. In particular, the present disclosure provides at least: structured update approaches in which the model update is restricted to be small and sketched update approaches in which the model update is compressed before sending to the server.
-
公开(公告)号：US12219004B2
公开(公告)日：2025-02-04
申请号：US18240799
申请日：2023-08-31
Applicant: Google LLC
Inventor： Ananda Theertha Suresh , Sanjiv Kumar , Hugh Brendan McMahan , Xinnan Yu
IPC: G06N20/00 , G06F17/12 , G06F17/16 , G06F17/18 , G06N7/01 , H03M7/30 , H03M7/40 , H04L67/10 , H04L67/01
Abstract: The present disclosure provides systems and methods for communication efficient distributed mean estimation. In particular, aspects of the present disclosure can be implemented by a system in which a number of vectors reside on a number of different clients, and a centralized server device seeks to estimate the mean of such vectors. According to one aspect of the present disclosure, a client computing device can rotate a vector by a random rotation matrix and then subsequently perform probabilistic quantization on the rotated vector. According to another aspect of the present disclosure, subsequent to quantization but prior to transmission, the client computing can encode the quantized vector according to a variable length coding scheme (e.g., by computing variable length codes).
-
-
-
-
-
-
-
-
-
Search Results
24
records
(took 0.014 seconds)
