公开(公告)号：US12205005B2

公开(公告)日：2025-01-21

申请号：US16931862

申请日：2020-07-17

Applicant: Google LLC

Inventor： Xinnan Yu ,  Ankit Singh Rawat ,  Jiecao Chen ,  Ananda Theertha Suresh ,  Sanjiv Kumar

IPC: G06N3/08 ,  G06F17/14 ,  G06F17/18 ,  G06F18/2431 ,  G06F40/20 ,  G06N3/084 ,  G06N20/00 ,  G06N20/10 ,  G06V10/77

Abstract: Systems and methods for low bias negative sampling of classes according to the sampled softmax method are described herein. The systems and methods can include training a machine-learned model for classifying inputs into one or more classes of a plurality of classes, each of the plurality of classes having an associated class embedding in a plurality of class embeddings. The systems and methods can include selecting, by the one or more computing devices, one or more negative classes from the plurality of classes based at least in part on a probability distribution approximating a softmax distribution, wherein the probability distribution is determined based at least in part on a Random Fourier Features map.

公开(公告)号：US20240311405A1

公开(公告)日：2024-09-19

申请号：US18337316

申请日：2023-06-19

Applicant: GOOGLE LLC

Inventor： Seungyeon Kim ,  Ankit Singh Rawat ,  Wittawat Jitkrittum ,  Hari Narasimhan ,  Sashank Reddi ,  Neha Gupta ,  Srinadh Bhojanapalli ,  Aditya Menon ,  Manzil Zaheer ,  Tal Schuster ,  Sanjiv Kumar ,  Toby Boyd ,  Zhifeng Chen ,  Emanuel Taropa ,  Vikram Kasivajhula ,  Trevor Strohman ,  Martin Baeuml ,  Leif Schelin ,  Yanping Huang

IPC: G06F16/332

CPC classification number: G06F16/3329

Abstract: Implementations disclose selecting, in response to receiving a request and from among multiple candidate generative models (e.g., multiple candidate large language models (LLMs)) with differing computational efficiencies, a particular generative model to utilize in generating a response to the request. Those implementations reduce latency and/or conserve computational resource(s) through selection, for various requests, of a more computationally efficient generative model for utilization in lieu of a less computationally efficient generative model. Further, those implementations seek to achieve such benefits, through utilization of more computationally efficient generative models, while also still selectively utilizing less computationally efficient generative models for certain requests to mitigate occurrences of a generated response being inaccurate and/or under-specified. This, in turn, can mitigate occurrences of computational and/or network inefficiencies that result from a user issuing a follow-up request to cure the inaccuracies and/or under-specification of a generated response.

公开(公告)号：US20230017505A1

公开(公告)日：2023-01-19

申请号：US17375960

申请日：2021-07-14

Applicant: Google LLC

Inventor： Aditya Krishna Menon ,  Sanjiv Kumar ,  Himanshu Jain ,  Andreas Veit ,  Ankit Singh Rawat ,  Gayan Sadeep Jayasumana Hirimbura Matara Kankanamge

IPC: G06N3/08 ,  G06K9/62

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for accounting for long-tail training data.

公开(公告)号：US20220335274A1

公开(公告)日：2022-10-20

申请号：US17721292

申请日：2022-04-14

Applicant: Google LLC

Inventor： Ankit Singh Rawat ,  Manzil Zaheer ,  Aditya Krishna Menon ,  Sanjiv Kumar ,  Amr Ahmed

IPC: G06N3/04 ,  G06K9/62

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for multi-stage computationally-efficient inference using a first and second neural network.

公开(公告)号：US20210019654A1

公开(公告)日：2021-01-21

申请号：US16931862

申请日：2020-07-17

Applicant: Google LLC

Inventor： Xinnan Yu ,  Ankit Singh Rawat ,  Jiecao Chen ,  Ananda Theertha Suresh ,  Sanjiv Kumar

IPC: G06N20/00 ,  G06F40/20 ,  G06K9/62 ,  G06F17/14 ,  G06F17/18

Abstract: Systems and methods for low bias negative sampling of classes according to the sampled softmax method are described herein. The systems and methods can include training a machine-learned model for classifying inputs into one or more classes of a plurality of classes, each of the plurality of classes having an associated class embedding in a plurality of class embeddings. The systems and methods can include selecting, by the one or more computing devices, one or more negative classes from the plurality of classes based at least in part on a probability distribution approximating a softmax distribution, wherein the probability distribution is determined based at least in part on a Random Fourier Features map.

公开(公告)号：US20240135254A1

公开(公告)日：2024-04-25

申请号：US18488951

申请日：2023-10-17

Applicant: Google LLC

Inventor： Harikrishna Narasimhan ,  Wittawat Jitkrittum ,  Aditya Krishna Menon ,  Ankit Singh Rawat ,  Sanjiv Kumar

IPC: G06N20/00 ,  G06N5/04

CPC classification number: G06N20/00 ,  G06N5/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for post-hoc deferral for classification tasks. In particular, a system can perform either post-hoc threshold correction or post-hoc rejector training to account for the cost of deferring model inputs to an expert system for classification.

公开(公告)号：US11676033B1

公开(公告)日：2023-06-13

申请号：US16812160

申请日：2020-03-06

Applicant: Google LLC

Inventor： Aditya Krishna Menon ,  Ankit Singh Rawat ,  Sashank Jakkam Reddi ,  Sanjiv Kumar

IPC: G06N3/04 ,  G06N3/084

CPC classification number: G06N3/084 ,  G06N3/04

Abstract: A method for training a machine learning model, e.g., a neural network, using a regularization scheme is disclosed. The method includes generating regularized partial gradients of losses computed using an objective function for training the machine learning model.

公开(公告)号：US20210326757A1

公开(公告)日：2021-10-21

申请号：US17227851

申请日：2021-04-12

Applicant: Google LLC

Inventor： Ankit Singh Rawat ,  Xinnan Yu ,  Aditya Krishna Menon ,  Sanjiv Kumar

IPC: G06N20/00 ,  G06F17/16 ,  G06N5/02

Abstract: Generally, the present disclosure is directed to systems and methods that perform spreadout regularization to enable learning of a multi-class classification model in the federated setting, where each user has access to the positive data associated with only a limited number of classes (e.g., a single class). Examples of such settings include decentralized training of face recognition models or speaker identification models, where in addition to the user specific facial images and voice samples, the class embeddings for the users also constitute sensitive information that cannot be shared with other users.

公开(公告)号：US20210319339A1

公开(公告)日：2021-10-14

申请号：US17227817

申请日：2021-04-12

Applicant: Google LLC

Inventor： Ankit Singh Rawat ,  Manzil Zaheer ,  Aditya Krishna Menon ,  Sanjiv Kumar ,  Melanie Weber

IPC: G06N5/04 ,  G06N20/00

Abstract: Generally, the present disclosure provides systems and methods for performing machine learning in hyperbolic space. Specifically, techniques are provided which enable the learning of a classifier (e.g., large-margin classifier) for data defined within a hyperbolic space (e.g., which may be particularly beneficial for data that possesses a hierarchical structure).