公开(公告)号：US20240311405A1

公开(公告)日：2024-09-19

申请号：US18337316

申请日：2023-06-19

Applicant: GOOGLE LLC

Inventor： Seungyeon Kim ,  Ankit Singh Rawat ,  Wittawat Jitkrittum ,  Hari Narasimhan ,  Sashank Reddi ,  Neha Gupta ,  Srinadh Bhojanapalli ,  Aditya Menon ,  Manzil Zaheer ,  Tal Schuster ,  Sanjiv Kumar ,  Toby Boyd ,  Zhifeng Chen ,  Emanuel Taropa ,  Vikram Kasivajhula ,  Trevor Strohman ,  Martin Baeuml ,  Leif Schelin ,  Yanping Huang

IPC: G06F16/332

CPC classification number: G06F16/3329

Abstract: Implementations disclose selecting, in response to receiving a request and from among multiple candidate generative models (e.g., multiple candidate large language models (LLMs)) with differing computational efficiencies, a particular generative model to utilize in generating a response to the request. Those implementations reduce latency and/or conserve computational resource(s) through selection, for various requests, of a more computationally efficient generative model for utilization in lieu of a less computationally efficient generative model. Further, those implementations seek to achieve such benefits, through utilization of more computationally efficient generative models, while also still selectively utilizing less computationally efficient generative models for certain requests to mitigate occurrences of a generated response being inaccurate and/or under-specified. This, in turn, can mitigate occurrences of computational and/or network inefficiencies that result from a user issuing a follow-up request to cure the inaccuracies and/or under-specification of a generated response.

公开(公告)号：US11886976B1

公开(公告)日：2024-01-30

申请号：US18222395

申请日：2023-07-14

Applicant: Google LLC

Inventor： Tal Schuster ,  Adam Joshua Fisch ,  Jai Prakash Gupta ,  Mostafa Dehghani ,  Dara Bahri ,  Vinh Quoc Tran ,  Yi Tay ,  Donald Arthur Metzler, Jr.

IPC: G06N3/0455

CPC classification number: G06N3/0455

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating output sequences using auto-regressive decoder neural networks. In particular, during generation, adaptive early exiting is used to reduce the time required to generate the output sequence.

公开(公告)号：US20240169184A1

公开(公告)日：2024-05-23

申请号：US18426212

申请日：2024-01-29

Applicant: Google LLC

Inventor： Tal Schuster ,  Adam Joshua Fisch ,  Jai Prakash Gupta ,  Mostafa Dehghani ,  Dara Bahri ,  Vinh Quoc Tran ,  Yi Tay ,  Donald Arthur Metzler, JR.

IPC: G06N3/0455

CPC classification number: G06N3/0455

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating output sequences using auto-regressive decoder neural networks. In particular, during generation, adaptive early exiting is used to reduce the time required to generate the output sequence.

公开(公告)号：US20240020516A1

公开(公告)日：2024-01-18

申请号：US18222395

申请日：2023-07-14

Applicant: Google LLC

Inventor： Tal Schuster ,  Adam Joshua Fisch ,  Jai Prakash Gupta ,  Mostafa Dehghani ,  Dara Bahri ,  Vinh Quoc Tran ,  Yi Tay ,  Donald Arthur Metzler, Jr.

IPC: G06N3/0455

CPC classification number: G06N3/0455

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating output sequences using auto-regressive decoder neural networks. In particular, during generation, adaptive early exiting is used to reduce the time required to generate the output sequence.