公开(公告)号：US20240330334A1

公开(公告)日：2024-10-03

申请号：US18225990

申请日：2023-07-25

Applicant: GOOGLE LLC

Inventor： Sidharth Mudgal ,  Ahmad Beirami ,  Jilin Chen ,  Alex Beutel ,  Harish Ganapathy ,  YaGuang Li ,  Tao Wang ,  Yanping Huang ,  Trevor Strohman

IPC: G06F16/332 ,  G06F40/284

CPC classification number: G06F16/3329 ,  G06F40/284

Abstract: Implementations relate to reducing latency in generating and/or rendering a given stream of natural language (NL) based output generated using a large language model (LLM). Processor(s) of a system can: receive NL based input associated with a client device, generate the stream of NL based output utilizing the LLM that is responsive to the NL based input and that is for a given dialog context of an ongoing dialog, and cause the stream of NL based output to be rendered at the client device. Notably, the processor(s) can employ attribute classifier(s) and a multi-objective scorer to implement a blockwise controlled decoding technique in generating the stream of NL based output utilizing the LLM. By implementing the blockwise controlled decoding technique in generating the stream of NL based output utilizing the LLM, the processor(s) can reduce latency in generating and/or of the stream of NL based output generated utilizing the LLM.

公开(公告)号：US20250095637A1

公开(公告)日：2025-03-20

申请号：US18886581

申请日：2024-09-16

Applicant: Google LLC

Inventor： Ke Hu ,  Tara N. Sainath ,  Bo Li ,  Yu Zhang ,  Yong Cheng ,  Tao Wang ,  Yujing Zhang ,  Frederick Liu

IPC: G10L15/06 ,  G10L15/00

Abstract: A method includes receiving a textual prompt in a first language and obtaining a fine-tuned prompt embedding configured to guide a large language model (LLM) to generate text in a target language from textual prompts in the first language. The method also includes processing, using the LLM, the textual prompt conditioned on the fine-tuned prompt embedding to generate output text in the target language and concatenating the textual prompt and the generated output text to provide an unspoken textual utterance. The method also includes training a multilingual automatic speech recognition (ASR) model to learn how to recognize speech in the target language by injecting the unspoken textual utterance into a text encoder associated with the multilingual ASR model.

公开(公告)号：US20240428006A1

公开(公告)日：2024-12-26

申请号：US18211967

申请日：2023-06-20

Applicant: GOOGLE LLC

Inventor： Jian Li ,  Zhifeng Chen ,  Yanping Huang ,  Yuanzhong Xu ,  Tao Wang ,  YaGuang Li

IPC: G06F40/40

Abstract: Implementations relate to asymmetric quantization of large language models (LLMs). Processor(s) of a system can: obtain a trained LLM, wherein the trained LLM includes a plurality of layers, each layer comprising a respective plurality of weights; for each layer of the plurality of layers: calculate an optimal clipping range for the respective plurality of weights, and clip one or more weights of the respective plurality of weights that lie outside of the optimal clipping range to produce a clipped layer; quantize the LLM to generate a quantized LLM, wherein the instructions to quantize include instructions to map weights of the plurality of clipped layers of the LLM from continuous values to discrete values; and provide the quantized LLM for downstream processing.

公开(公告)号：US20240303464A1

公开(公告)日：2024-09-12

申请号：US18598876

申请日：2024-03-07

Applicant: Google LLC

Inventor： Nan Du ,  Tao Wang ,  Yanqi Zhou ,  Tao Lei ,  Yuanzhong Xu ,  Andrew Mingbo Dai ,  Zhifeng Chen ,  Dewen Zeng ,  Yingwei Cui

IPC: G06N3/04 ,  G06N3/084

CPC classification number: G06N3/04 ,  G06N3/084

Abstract: A method includes providing a first set of data objects to a first skip router of a neural network (NN). The NN includes a first NN layer and a second NN layer. The first set of data objects is subdivided into a first set of skip objects and a first set of non-skip objects based on a first skip logic implemented by the first skip router and a first context of each data object in the first set of data objects. A first set of processed objects is generated based on the first set of non-skip objects and a first layer logic implemented by the first NN layer. Predictions are generated based on a second set of data objects and a second layer logic implemented by the second NN layer. The second set of data objects includes the first set of processed objects and the first set of skip objects.

公开(公告)号：US20240282294A1

公开(公告)日：2024-08-22

申请号：US18651296

申请日：2024-04-30

Applicant: Google LLC

Inventor： Qingqing Huang ,  Daniel Sung-Joon Park ,  Aren Jansen ,  Timo Immanuel Denk ,  Yue Li ,  Ravi Ganti ,  Dan Ellis ,  Tao Wang ,  Wei Han ,  Joonseok Lee

IPC: G10L15/06 ,  G10L15/16

CPC classification number: G10L15/063 ,  G10L15/16

Abstract: A corpus of textual data is generated with a machine-learned text generation model. The corpus of textual data includes a plurality of sentences. Each sentence is descriptive of a type of audio. For each of a plurality of audio recordings, the audio recording is processed with a machine-learned audio classification model to obtain training data including the audio recording and one or more sentences of the plurality of sentences closest to the audio recording within a joint audio-text embedding space of the machine-learned audio classification model. The sentence(s) are processed with a machine-learned generation model to obtain an intermediate representation of the one or more sentences. The intermediate representation is processed with a machine-learned cascaded diffusion model to obtain audio data. The machine-learned cascaded diffusion model is trained based on a difference between the audio data and the audio recording.