公开(公告)号：US20240232637A9

公开(公告)日：2024-07-11

申请号：US18491877

申请日：2023-10-23

Applicant: Google LLC

Inventor： Krishna Pragash Srinivasan ,  Michael Bendersky ,  Anupam Samanta ,  Lingrui Liao ,  Luca Bertelli ,  Ming-Wei Chang ,  Iftekhar Naim ,  Siddhartha Brahma ,  Siamak Shakeri ,  Hongkun Yu ,  John Nham ,  Karthik Raman ,  Raphael Dominik Hoffmann

IPC: G06N3/0895 ,  G06F16/903 ,  G06F16/93 ,  G06N3/0455

CPC classification number: G06N3/0895 ,  G06F16/90335 ,  G06F16/93 ,  G06N3/0455

Abstract: Provided are computing systems, methods, and platforms that train query processing models, such as large language models, to perform query intent classification tasks by using retrieval augmentation and multi-stage distillation. Unlabeled training examples of queries may be obtained, and a set of the training examples may be augmented with additional feature annotations to generate augmented training examples. A first query processing model may annotate the retrieval augmented queries to generate inferred labels for the augmented training examples. A second query processing model may be trained on the inferred labels, distilling the query processing model that was trained with retrieval augmentation into a non-retrieval augmented query processing model. The second query processing model may annotate the entire set of unlabeled training examples. Another stage of distillation may train a third query processing model using the entire set of unlabeled training examples without retrieval augmentation.

公开(公告)号：US20240135187A1

公开(公告)日：2024-04-25

申请号：US18491877

申请日：2023-10-22

Applicant: Google LLC

Inventor： Krishna Pragash Srinivasan ,  Michael Bendersky ,  Anupam Samanta ,  Lingrui Liao ,  Luca Bertelli ,  Ming-Wei Chang ,  Iftekhar Naim ,  Siddhartha Brahma ,  Siamak Shakeri ,  Hongkun Yu ,  John Nham ,  Karthik Raman ,  Raphael Dominik Hoffmann

IPC: G06N3/0895 ,  G06F16/903 ,  G06F16/93 ,  G06N3/0455

CPC classification number: G06N3/0895 ,  G06F16/90335 ,  G06F16/93 ,  G06N3/0455

Abstract: Provided are computing systems, methods, and platforms that train query processing models, such as large language models, to perform query intent classification tasks by using retrieval augmentation and multi-stage distillation. Unlabeled training examples of queries may be obtained, and a set of the training examples may be augmented with additional feature annotations to generate augmented training examples. A first query processing model may annotate the retrieval augmented queries to generate inferred labels for the augmented training examples. A second query processing model may be trained on the inferred labels, distilling the query processing model that was trained with retrieval augmentation into a non-retrieval augmented query processing model. The second query processing model may annotate the entire set of unlabeled training examples. Another stage of distillation may train a third query processing model using the entire set of unlabeled training examples without retrieval augmentation.