公开(公告)号：US20240378394A1

公开(公告)日：2024-11-14

申请号：US18231650

申请日：2023-08-08

Applicant: GOOGLE LLC

Inventor： Ragha Kotikalapudi ,  Chen Zhu ,  Steven Zheng ,  Sahitya Potluri ,  Yu Du ,  Heng-Tze Cheng ,  Quoc Le ,  Ed H. Chi

IPC: G06F40/40 ,  G06N3/09

Abstract: Implementations described herein relate to using self-evaluation when utilizing a large language model (LLM) to generate a response to a natural language (NL) based input. The LLM can be used to process the NL based input to generate a plurality of responses, and to generate a critique of those responses by comparing the responses to a set of response evaluation criteria. One of the responses can then be selected based on the comparison with the set of response evaluation criteria which can vary from one NL based input to another. If the NL based input was obtained a user of a client device during an inference stage, then the selected response can be rendered for presentation to the user. If the NL based input was obtained during a training stage, then the selected response can be stored as a training instance and optionally in association with additional data.

公开(公告)号：US20240289395A1

公开(公告)日：2024-08-29

申请号：US18528142

申请日：2023-12-04

Applicant: Google LLC

Inventor： Hao Zhou ,  Shrestha Basu Mallick ,  Trevor Strohman ,  Patricia Luisa Romero Domingo ,  Amirhossein Kiani ,  Yu Du ,  Xinying Song ,  Heng-Tze Cheng ,  Quoc V. Le ,  Ed Huai-Hsin Chi ,  Christopher Jamie Maclean Hall

IPC: G06F16/9532 ,  G06F16/955 ,  G06F40/40

CPC classification number: G06F16/9532 ,  G06F16/955 ,  G06F40/40

Abstract: Implementations relate to helping a large language model generate factual responses to prompts that request factual content is disclosed. The large language model may receive a prompt context, a plurality of encoded context passages as input. The large language model is trained to determine whether or not to utilize the encoded context passages in generating the response. Implementations also relate to different methods of fine-tuning the responses generated by the large language model through query refinements, response re-writes, and evaluation of factual accuracy.

公开(公告)号：US11907674B1

公开(公告)日：2024-02-20

申请号：US18370683

申请日：2023-09-20

Applicant: GOOGLE LLC

Inventor： Oscar Akerlund ,  Evgeny Sluzhaev ,  Golnaz Ghiasi ,  Thang Luong ,  Yifeng Lu ,  Igor Petrovski ,  Ágoston Weisz ,  Wei Yu ,  Rakesh Shivanna ,  Michael Andrew Goodman ,  Apoorv Kulshreshtha ,  Yu Du ,  Amin Ghafouri ,  Sanil Jain ,  Dustin Tran ,  Vikas Peswani ,  YaGuang Li

IPC: G06F40/00 ,  G06F40/40

CPC classification number: G06F40/40

Abstract: Implementations relate to generating multi-modal response(s) through utilization of large language model(s) (LLM(s)). Processor(s) of a system can: receive natural language (NL) based input, generate a multi-modal response that is responsive to the NL based output, and cause the multi-modal response to be rendered. In some implementations, and in generating the multi-modal response, the processor(s) can process, using a LLM, LLM input (e.g., that includes at least the NL based input) to generate LLM output, and determine, based on the LLM output, textual content for inclusion in the multi-modal response and multimedia content for inclusion in the multi-modal response. In some implementations, the multimedia content can be obtained based on a multimedia content tag that is included in the LLM output and that is indicative of the multimedia content. In various implementations, the multimedia content can be interleaved between segments of the textual content.

公开(公告)号：US20250053751A1

公开(公告)日：2025-02-13

申请号：US18413495

申请日：2024-01-16

Applicant: GOOGLE LLC

Inventor： Oscar Akerlund ,  Evgeny Sluzhaev ,  Golnaz Ghiasi ,  Thang Luong ,  Yifeng Lu ,  Igor Petrovski ,  Agoston Weisz ,  Wei Yu ,  Rakesh Shivanna ,  Michael Andrew Goodman ,  Apoorv Kulshreshtha ,  Yu Du ,  Amin Ghafouri ,  Sanil Jain ,  Dustin Tran ,  Vikas Peswani ,  YaGuang Li

IPC: G06F40/40

Abstract: Implementations relate to generating multi-modal response(s) through utilization of large language model(s) (LLM(s)). Processor(s) of a system can: receive natural language (NL) based input, generate a multi-modal response that is responsive to the NL based output, and cause the multi-modal response to be rendered. In some implementations, and in generating the multi-modal response, the processor(s) can process, using a LLM, LLM input (e.g., that includes at least the NL based input) to generate LLM output, and determine, based on the LLM output, textual content for inclusion in the multi-modal response and multimedia content for inclusion in the multi-modal response. In some implementations, the multimedia content can be obtained based on a multimedia content tag that is included in the LLM output and that is indicative of the multimedia content. In various implementations, the multimedia content can be interleaved between segments of the textual content.

公开(公告)号：US20250045534A1

公开(公告)日：2025-02-06

申请号：US18378434

申请日：2023-10-10

Applicant: GOOGLE LLC

Inventor： Swaroop Mishra ,  Ragha Kotikalapudi ,  Sahitya Potluri ,  Taylor Bos ,  YaGuang Li ,  Hanzhao Lin ,  Steven Zheng ,  Yu Du ,  Chen Zhu ,  Chenkai Kuang ,  Xinying Song ,  Heng-Tze Cheng ,  Ed H. Chi ,  Quoc Le

IPC: G06F40/40

Abstract: Implementations relate to a method implemented by one or more processors, the method including: receiving natural language (NL) based input associated with a client device; generating, using a large language model (LLM) and based on processing the NL based input, LLM output; determining, based on the LLM output, a sequence of LLM responses, the sequence of LLM responses including at least one intermediate LLM response and a final LLM response. In some implementations, the method may further include causing the final LLM response to be rendered at the client device. In additional or alternative implementations, the method may further include storing, as an instance of training data for fine-tuning the LLM or an additional LLM, the NL based input along with the final LLM response.

公开(公告)号：US20240394471A1

公开(公告)日：2024-11-28

申请号：US18231586

申请日：2023-08-08

Applicant: GOOGLE LLC

Inventor： Ragha Kotikalapudi ,  Swaroop Mishra ,  Sahitya Potluri ,  Taylor Bos ,  Yu Du ,  Chen Zhu ,  Steven Zheng ,  Hanzhao Lin ,  Summer Yue ,  Heng-Tze Cheng ,  Quoc Le ,  Ed H. Chi

IPC: G06F40/20

Abstract: Implementations relate to improving instruction following capabilities of large language models (LLMs) using instruction decomposition, self-evaluation, and optionally progressive refinement. Processor(s) of a system can: obtain natural language (NL) based input, generate a plurality of candidate responses and evaluate the candidate responses based on instructions included in the NL based input, using an LLM, and progressively refine the candidate responses until it is determined that one or more termination criteria are satisfied. In some implementations, the NL based input can be received from a client device. In these implementations, a given candidate response that is progressively refined can be rendered for presentation at the client device and responsive to the NL base input. In additional or alternative implementations, the NL based input can be obtained from database(s). In these implementations, a given candidate response that is progressively refined can be utilized in fine-tuning of the LLM.

公开(公告)号：US20240362093A1

公开(公告)日：2024-10-31

申请号：US18231606

申请日：2023-08-08

Applicant: GOOGLE LLC

Inventor： Hao Zhou ,  Jamie Hall ,  Xinying Song ,  Sahitya Potluri ,  Yu Du ,  Heng-Tze Cheng ,  Quoc Le ,  Ed H. Chi

IPC: G06F9/54 ,  G06F16/242

CPC classification number: G06F9/547 ,  G06F16/243

Abstract: At least utilizing a custom corpus of documents to condition a large language model (LLM) when generating a response to a user query. In some implementations, a user query associated with a client device is received. An API query for an external application is generated by an LLM based on the user query. The external application has access to a custom corpus of documents comprising a plurality of documents. The external application is queried using the API query. Data representative of one or more documents in the custom corpus of documents is received from the external application in response to the API query. The LLM generates a response to the query that is conditioned on the data representing one or more of the documents in the custom corpus of documents received from the external application. The response to the user query is caused to be rendered on the client device.