Invention Publication
- Patent Title: Method for Training Large Language Models to Perform Query Intent Classification
-
Application No.: US18491877Application Date: 2023-10-22
-
Publication No.: US20240135187A1Publication Date: 2024-04-25
- Inventor: Krishna Pragash Srinivasan , Michael Bendersky , Anupam Samanta , Lingrui Liao , Luca Bertelli , Ming-Wei Chang , Iftekhar Naim , Siddhartha Brahma , Siamak Shakeri , Hongkun Yu , John Nham , Karthik Raman , Raphael Dominik Hoffmann
- Applicant: Google LLC
- Applicant Address: US CA Mountain View
- Assignee: Google LLC
- Current Assignee: Google LLC
- Current Assignee Address: US CA Mountain View
- Main IPC: G06N3/0895
- IPC: G06N3/0895 ; G06F16/903 ; G06F16/93 ; G06N3/0455

Abstract:
Provided are computing systems, methods, and platforms that train query processing models, such as large language models, to perform query intent classification tasks by using retrieval augmentation and multi-stage distillation. Unlabeled training examples of queries may be obtained, and a set of the training examples may be augmented with additional feature annotations to generate augmented training examples. A first query processing model may annotate the retrieval augmented queries to generate inferred labels for the augmented training examples. A second query processing model may be trained on the inferred labels, distilling the query processing model that was trained with retrieval augmentation into a non-retrieval augmented query processing model. The second query processing model may annotate the entire set of unlabeled training examples. Another stage of distillation may train a third query processing model using the entire set of unlabeled training examples without retrieval augmentation.
Public/Granted literature
- US20240232637A9 Method for Training Large Language Models to Perform Query Intent Classification Public/Granted day:2024-07-11
Information query