公开(公告)号：US12001509B2

公开(公告)日：2024-06-04

申请号：US16821509

申请日：2020-03-17

Applicant: Google LLC

Inventor： Seungyeon Kim ,  Jingzhao Zhang ,  Andreas Veit ,  Sanjiv Kumar ,  Sashank Reddi ,  Praneeth Karimireddy

IPC: G06N20/00 ,  G06F17/18 ,  G06F18/21 ,  G06N3/084

CPC classification number: G06F17/18 ,  G06F18/217 ,  G06N20/00 ,  G06N3/084

Abstract: Generally, the present disclosure is directed to systems and methods that perform adaptive optimization with improved convergence properties. The adaptive optimization techniques described herein are useful in various optimization scenarios, including, for example, training a machine-learned model such as, for example, a neural network. In particular, according to one aspect of the present disclosure, a system implementing the adaptive optimization technique can, over a plurality of iterations, employ an adaptive per coordinate clipping threshold to clip a current first moment of the coordinate to obtain a current update value that enables faster convergence for the machine-learned model when the noise in the stochastic gradients is heavy tailed.

公开(公告)号：US11803290B2

公开(公告)日：2023-10-31

申请号：US17156972

申请日：2021-01-25

Applicant: Google LLC

Inventor： Tim Wantland ,  Julian Odell ,  Seungyeon Kim ,  Iulia Turc ,  Daniel Ramage ,  Wei Huang ,  Kaikai Wang

IPC: G06F17/00 ,  G06F3/0482 ,  G06F3/0484

CPC classification number: G06F3/0482 ,  G06F3/0484

Abstract: The present disclosure is directed to input suggestion. In particular, the methods and systems of the present disclosure can: receive, from a first application executed by one or more computing devices, data indicating information that has been presented by and/or input into the first application; generate, based at least in part on the received data, one or more suggested candidate inputs for a second application executed by the computing device(s); provide, in association with the second application, an interface comprising one or more options to select at least one suggested candidate input of the suggested candidate input(s); and responsive to receiving data indicating a selection of a particular suggested candidate input of the suggested candidate input(s) via the interface, communicate, to the second application, data indicating the particular suggested candidate input.

公开(公告)号：US20240311405A1

公开(公告)日：2024-09-19

申请号：US18337316

申请日：2023-06-19

Applicant: GOOGLE LLC

Inventor： Seungyeon Kim ,  Ankit Singh Rawat ,  Wittawat Jitkrittum ,  Hari Narasimhan ,  Sashank Reddi ,  Neha Gupta ,  Srinadh Bhojanapalli ,  Aditya Menon ,  Manzil Zaheer ,  Tal Schuster ,  Sanjiv Kumar ,  Toby Boyd ,  Zhifeng Chen ,  Emanuel Taropa ,  Vikram Kasivajhula ,  Trevor Strohman ,  Martin Baeuml ,  Leif Schelin ,  Yanping Huang

IPC: G06F16/332

CPC classification number: G06F16/3329

Abstract: Implementations disclose selecting, in response to receiving a request and from among multiple candidate generative models (e.g., multiple candidate large language models (LLMs)) with differing computational efficiencies, a particular generative model to utilize in generating a response to the request. Those implementations reduce latency and/or conserve computational resource(s) through selection, for various requests, of a more computationally efficient generative model for utilization in lieu of a less computationally efficient generative model. Further, those implementations seek to achieve such benefits, through utilization of more computationally efficient generative models, while also still selectively utilizing less computationally efficient generative models for certain requests to mitigate occurrences of a generated response being inaccurate and/or under-specified. This, in turn, can mitigate occurrences of computational and/or network inefficiencies that result from a user issuing a follow-up request to cure the inaccuracies and/or under-specification of a generated response.

公开(公告)号：US20210295201A1

公开(公告)日：2021-09-23

申请号：US16821509

申请日：2020-03-17

Applicant: Google LLC

Inventor： Seungyeon Kim ,  Jingzhao Zhang ,  Andreas Veit ,  Sanjiv Kumar ,  Sashank Reddi ,  Praneeth Karimireddy

IPC: G06N20/00 ,  G06K9/62 ,  G06F17/18

Abstract: Generally, the present disclosure is directed to systems and methods that perform adaptive optimization with improved convergence properties. The adaptive optimization techniques described herein are useful in various optimization scenarios, including, for example, training a machine-learned model such as, for example, a neural network. In particular, according to one aspect of the present disclosure, a system implementing the adaptive optimization technique can, over a plurality of iterations, employ an adaptive per coordinate clipping threshold to clip a current first moment of the coordinate to obtain a current update value that enables faster convergence for the machine-learned model when the noise in the stochastic gradients is heavy tailed.

公开(公告)号：US10901577B2

公开(公告)日：2021-01-26

申请号：US16037418

申请日：2018-07-17

Applicant: Google LLC

Inventor： Tim Wantland ,  Julian Odell ,  Seungyeon Kim ,  Iulia Turc ,  Daniel Ramage ,  Wei Huang ,  Kaikai Wang

IPC: G06F17/00 ,  G06F3/0482 ,  G06F3/0484

Abstract: The present disclosure is directed to input suggestion. In particular, the methods and systems of the present disclosure can: receive, from a first application executed by one or more computing devices, data indicating information that has been presented by and/or input into the first application; generate, based at least in part on the received data, one or more suggested candidate inputs for a second application executed by the computing device(s); provide, in association with the second application, an interface comprising one or more options to select at least one suggested candidate input of the suggested candidate input(s); and responsive to receiving data indicating a selection of a particular suggested candidate input of the suggested candidate input(s) via the interface, communicate, to the second application, data indicating the particular suggested candidate input.

公开(公告)号：US20200026395A1

公开(公告)日：2020-01-23

申请号：US16037418

申请日：2018-07-17

Applicant: Google LLC

Inventor： Tim Wantland ,  Julian Odell ,  Seungyeon Kim ,  Iulia Turc ,  Daniel Ramage ,  Wei Huang ,  Kaikai Wang

IPC: G06F3/0482 ,  G06F3/0484

Abstract: The present disclosure is directed to input suggestion. In particular, the methods and systems of the present disclosure can: receive, from a first application executed by one or more computing devices, data indicating information that has been presented by and/or input into the first application; generate, based at least in part on the received data, one or more suggested candidate inputs for a second application executed by the computing device(s); provide, in association with the second application, an interface comprising one or more options to select at least one suggested candidate input of the suggested candidate input(s); and responsive to receiving data indicating a selection of a particular suggested candidate input of the suggested candidate input(s) via the interface, communicate, to the second application, data indicating the particular suggested candidate input.

公开(公告)号：US20240012540A1

公开(公告)日：2024-01-11

申请号：US18471966

申请日：2023-09-21

Applicant: Google LLC

Inventor： Tim Wantland ,  Julian Odell ,  Seungyeon Kim ,  Iulia Turc ,  Daniel Ramage ,  Wei Huang ,  Kaikai Wang

IPC: G06F3/0482 ,  G06F3/0484

CPC classification number: G06F3/0482 ,  G06F3/0484

Abstract: The present disclosure is directed to input suggestion. In particular, the methods and systems of the present disclosure can: receive, from a first application executed by one or more computing devices, data indicating information that has been presented by and/or input into the first application; generate, based at least in part on the received data, one or more suggested candidate inputs for a second application executed by the computing device(s); provide, in association with the second application, an interface comprising one or more options to select at least one suggested candidate input of the suggested candidate input(s); and responsive to receiving data indicating a selection of a particular suggested candidate input of the suggested candidate input(s) via the interface, communicate, to the second application, data indicating the particular suggested candidate input.

公开(公告)号：US20210141501A1

公开(公告)日：2021-05-13

申请号：US17156972

申请日：2021-01-25

Applicant: Google LLC

Inventor： Tim Wantland ,  Julian Odell ,  Seungyeon Kim ,  Iulia Turc ,  Daniel Ramage ,  Wei Huang ,  Kaikai Wang

IPC: G06F3/0482 ,  G06F3/0484

Abstract: The present disclosure is directed to input suggestion. In particular, the methods and systems of the present disclosure can: receive, from a first application executed by one or more computing devices, data indicating information that has been presented by and/or input into the first application; generate, based at least in part on the received data, one or more suggested candidate inputs for a second application executed by the computing device(s); provide, in association with the second application, an interface comprising one or more options to select at least one suggested candidate input of the suggested candidate input(s); and responsive to receiving data indicating a selection of a particular suggested candidate input of the suggested candidate input(s) via the interface, communicate, to the second application, data indicating the particular suggested candidate input.