公开(公告)号：US20230274143A1

公开(公告)日：2023-08-31

申请号：US18173985

申请日：2023-02-24

Applicant: Google LLC

Inventor： Zizhao Zhang ,  Zifeng Wang ,  Chen-Yu Lee ,  Ruoxi Sun ,  Sayna Ebrahimi ,  Xiaoqi Ren ,  Guolong Su ,  Vincent Perot ,  Tomas Pfister ,  Han Zhang

IPC: G06N3/08

CPC classification number: G06N3/08

Abstract: A method for rehearsal-free continual learning includes obtaining a set of training samples where training sample in the set of training samples is associated with a respective task of a plurality of different tasks. The method includes obtaining a task-invariant prompt representative of learned knowledge common to each respective task of the plurality of different tasks. The method includes, for each respective task of the plurality of different tasks, obtaining a respective task-specific prompt representative of learned knowledge specific to the respective task. The method includes, during each of one or more training iterations, for each respective training sample in the set of training samples, selecting the respective task-specific prompt representative of the respective task of the respective training sample and training a model using the task-invariant prompt and the selected respective task-specific prompt.

公开(公告)号：US20230059708A1

公开(公告)日：2023-02-23

申请号：US17797966

申请日：2021-02-08

Applicant: Google LLC

Inventor： Luke Shekerjian Metz ,  Ruoxi Sun ,  Christian Daniel Freeman ,  Benjamin Michael Poole ,  Niru Maheswaranathan ,  Jascha Narain Sohl-Dickstein

IPC: G06N3/08

Abstract: The present disclosure provides a computer-implemented method for determining an optimized list of sets of hyperparameter values for application to an additional machine learning task. The method includes obtaining data describing a plurality of different machine learning tasks. The method includes obtaining a plurality of candidate sets of hyperparameter values. The method includes determining an ordered list of sets of hyperparameters selected from the plurality of candidate sets of hyperparameter values, wherein the ordered list of sets of hyperparameters minimizes an aggregate loss over the plurality of different machine learning tasks. The method includes storing the ordered list of sets of hyperparameters for use in training an additional machine learning model to perform an additional machine learning task.

公开(公告)号：US20240394545A1

公开(公告)日：2024-11-28

申请号：US18377368

申请日：2023-10-06

Applicant: Google LLC

Inventor： Julian Martin Eisenschlos ,  Xingchen Wan ,  Hootan Nakhost ,  Sercan Omer Arik ,  Ruoxi Sun ,  Hanjun Dai

IPC: G06N3/088 ,  G06N3/0455

Abstract: Aspects of the disclosure are directed to methods, systems, and computer readable media for universal self-adaptive prompting (USP), which includes an automatic prompt design approach specifically tailored for zero-shot learning, though still compatible with few-shot learning. To achieve universal prompting, USP categorizes a natural language processing (NLP) task into one of a plurality of possible task types and then uses a corresponding selector to select the most suitable queries and zero-shot model-generated responses as pseudo-demonstrations, thereby generalizing in-context learning to the zero-shot setup in a fully automated manner.

公开(公告)号：US20240362212A1

公开(公告)日：2024-10-31

申请号：US18225277

申请日：2023-07-24

Applicant: Google LLC

Inventor： Ruoxi Sun ,  Sercan Omer Arik ,  Rajarishi Sinha ,  Hootan Nakhost ,  Hanjun Dai ,  Pengcheng Yin

IPC: G06F16/2452 ,  G06F16/242

CPC classification number: G06F16/24522 ,  G06F16/2433

Abstract: Aspects of the disclosure are directed to methods, systems, and non-transitory computer readable media for automatically generating queries on a database from natural language text using in-context learning to leverage zero-shot and few-shot adaptation capabilities of large language models (LLMs). The methods, systems, and non-transitory computer readable media can consider database information, employ execution based consistency decoding, and employ a mixture of prompts and/or LLMs.

公开(公告)号：US20240249080A1

公开(公告)日：2024-07-25

申请号：US18128450

申请日：2023-03-30

Applicant: Google LLC

Inventor： Ruoxi Sun ,  Xingchen Wan ,  Hanjun Dai ,  Sercan Omer Arik ,  Tomas Pfister

IPC: G06F40/40 ,  G06F16/33

CPC classification number: G06F40/40 ,  G06F16/3344

Abstract: Aspects of the disclosure are directed to automatically selecting examples in a prompt for an LLM to demonstrate how to perform tasks. Aspects of the disclosure can select and build a set of examples from LLM zero-shot outputs via predetermined criteria that can combine consistency, diversity, and repetition. In the zero-shot setting for three different LLMs, using only LLM predictions, aspects of the disclosure can improve performance up to 15% compared to zero-shot baselines and can match or exceed few-shot base-lines for a range of reasoning tasks.