公开(公告)号：US20210089828A1

公开(公告)日：2021-03-25

申请号：US17030316

申请日：2020-09-23

Applicant: Google LLC

Inventor： Sercan Omer Arik ,  Jinsung Yoon ,  Tomas Jon Pfister

IPC: G06K9/62 ,  G06N3/02

Abstract: A method for training a locally interpretable model includes obtaining a set of training samples and training a black-box model using the set of training samples. The method also includes generating, using the trained black-box model and the set of training samples, a set of auxiliary training samples and training a baseline interpretable model using the set of auxiliary training samples. The method also includes training, using the set of auxiliary training samples and baseline interpretable model, an instance-wise weight estimator model. For each auxiliary training sample in the set of auxiliary training samples, the method also includes determining, using the trained instance-wise weight estimator model, a selection probability for the auxiliary training sample. The method also includes selecting, based on the selection probabilities, a subset of auxiliary training samples and training the locally interpretable model using the subset of auxiliary training samples.

公开(公告)号：US20250156456A1

公开(公告)日：2025-05-15

申请号：US18947752

申请日：2024-11-14

Applicant: Google LLC

Inventor： Ruoxi Sun ,  Xi Ye ,  Sercan Omer Arik

IPC: G06F16/332 ,  G06F40/30

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for grounding LLMs. In one aspect, a method includes accessing responses from a large language model, each response comprising data including: a query, an answer to the query, the answer comprising one or more statements, citations linking each statement to an evidence passage in a corpus; determining a grounding quality of the answer based on the evidence passages and the statements using an attribution evaluation model, wherein the grounding quality is a quantification of attribution of the statements in the answer to a document corpus; and tuning the large language model to obtain an adapted large language model that satisfies a grounding constraint based on grounding quality score.

公开(公告)号：US12271822B2

公开(公告)日：2025-04-08

申请号：US17000094

申请日：2020-08-21

Applicant: Google LLC

Inventor： Zizhao Zhang ,  Tomas Jon Pfister ,  Sercan Omer Arik ,  Mingfei Gao

IPC: G06N3/044 ,  G06F7/24 ,  G06F18/211 ,  G06F18/214 ,  G06N3/045 ,  G06N3/08 ,  G06N3/084 ,  G06N7/01 ,  G06N20/00

Abstract: A method for active learning includes obtaining a set of unlabeled training samples and for each unlabeled training sample, perturbing the unlabeled training sample to generate an augmented training sample. The method includes generating, using a machine learning model, a predicted label for both samples and determining an inconsistency value for the unlabeled training sample that represents variance between the predicted labels for the unlabeled and augmented training samples. The method includes sorting the unlabeled training samples based on the inconsistency values and obtaining, for a threshold number of samples selected from the sorted unlabeled training samples, a ground truth label. The method includes selecting a current set of labeled training samples including each selected unlabeled training samples paired with the corresponding ground truth label. The method includes training, using the current set and a proper subset of unlabeled training samples, the machine learning model.

公开(公告)号：US20250111285A1

公开(公告)日：2025-04-03

申请号：US18902137

申请日：2024-09-30

Applicant: Google LLC

Inventor： Yan Liu ,  Chuizheng Meng ,  Yihe Dong ,  Sercan Omer Arik ,  Tomas Pfister

IPC: G06N20/00

Abstract: A machine-learned model includes an encoder having a feature block configured to embed input data into a plurality of features in an embedding space. The input data includes multiple components such as covariate, treatment, and output components. The encoder includes one or more encoding layers, each including a temporal attention block and a feature-wise attention block. The temporal attention block is configured to obtain the embedded input data and apply temporal causal attention along a time dimension in parallel for each feature of the plurality of features to generate temporal embeddings. The feature-wise attention block is configured to obtain the temporal embeddings and generate component representations such as a covariate representation, a treatment representation, and an output representation.

公开(公告)号：US20240386321A1

公开(公告)日：2024-11-21

申请号：US18639519

申请日：2024-04-18

Applicant: Google LLC

Inventor： Sayna Ebrahimi ,  Yihe Dong ,  Tomas Pfister ,  Sercan Omer Arik

IPC: G06N20/00

Abstract: Aspects of the disclosure are directed to a multimodal processing system for processing both structured and un-structured data. Real-world data is not always consistent in form or content. The multimodal processing system includes model that can be trained to account for this characteristic of real-world data, by selectively masking data of different modalities during pretraining to learn outputs that are the same or comparable between the masked and un-masked inputs. The model is trained according to modality-specific masking objectives computed for each modality of data and joint modality similarity-based masking objectives for a joint representation of the data across all modalities. The system provides consistent and accurate input, even when input data may have substantial portions of data from different modalities missing. Cross-modal relationships in data are reinforced by the model as different portions of data are masked, contributing to an overall increase in model accuracy versus other approaches.

公开(公告)号：US12039443B2

公开(公告)日：2024-07-16

申请号：US18045722

申请日：2022-10-11

Applicant: Google LLC

Inventor： Sercan Omer Arik ,  Chen Xing ,  Zizhao Zhang ,  Tomas Jon Pfister

IPC: G06N3/08 ,  G06F18/214 ,  G06F18/2413 ,  G06F18/2431 ,  G06N3/04

CPC classification number: G06N3/08 ,  G06F18/2148 ,  G06F18/2413 ,  G06F18/2431 ,  G06N3/04

Abstract: A method includes receiving a training data set including a plurality of training data subsets. From two or more training data subsets in the training data set, the method includes selecting a support set of training examples and a query set of training examples. The method includes determining, using the classification model, a centroid value for each respective class. For each training example in the query set of training examples, the method includes generating, using the classification model, a query encoding, determining a class distance measure, determining a ground-truth distance, and updating parameters of the classification model. For each training example in the query set of training examples identified as being misclassified, the method further includes generating a standard deviation value, sampling a new query, and updating parameters of the confidence model based on the new query encoding.

公开(公告)号：US12026614B2

公开(公告)日：2024-07-02

申请号：US16945898

申请日：2020-08-02

Applicant: Google LLC

Inventor： Sercan Omer Arik ,  Tomas Jon Pfister

IPC: G06N3/08 ,  G06N3/04

CPC classification number: G06N3/08 ,  G06N3/04

Abstract: A method of interpreting tabular data includes receiving, at a deep tabular data learning network (TabNet) executing on data processing hardware, a set of features. For each of multiple sequential processing steps, the method also includes: selecting, using a sparse mask of the TabNet, a subset of relevant features of the set of features; processing using a feature transformer of the TabNet, the subset of relevant features to generate a decision step output and information for a next processing step in the multiple sequential processing steps; and providing the information to the next processing step. The method also includes determining a final decision output by aggregating the decision step outputs generated for the multiple sequential processing steps.

公开(公告)号：US20240112084A1

公开(公告)日：2024-04-04

申请号：US18372900

申请日：2023-09-26

Applicant: Google LLC

Inventor： Sercan Omer Arik ,  Yihe Dong

IPC: G06N20/00

CPC classification number: G06N20/00

Abstract: Aspects of the disclosure are directed to a canonical approach for feature selection referred to as sparse learnable masks (SLM). SLM integrates learnable sparse masks into end-to-end training. For the fundamental non-differentiability challenge of selecting a desired number of features, SLM includes dual mechanisms for automatic mask scaling by achieving a desired feature sparsity and gradually tempering this sparsity for effective learning. SLM further employs an objective that increases mutual information (MI) between selected features and labels in an efficient and scalable manner. Empirically, SLM can achieve or improve upon state-of-the-art results on several benchmark datasets, often by a significant margin, while reducing computational complexity and cost.

公开(公告)号：US11941531B1

公开(公告)日：2024-03-26

申请号：US16785032

申请日：2020-02-07

Applicant: Google LLC

Inventor： Sercan Omer Arik ,  Tomas Jon Pfister

IPC: G06N3/088 ,  G06F40/30 ,  G06N3/045

CPC classification number: G06N3/088 ,  G06F40/30 ,  G06N3/045

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for processing an input data element to generate a prediction output that characterizes the input data element. In one aspect, a method comprises: determining a respective attention weight between an input data element and each of a plurality of reference data elements; processing each of the reference data elements using the encoder neural network to generate a respective value embedding of each reference data element; determining a combined value embedding of the reference data elements based on (i) the respective value embedding of each reference data element, and (ii) the respective attention weight between the input data element and each reference data element; and processing the combined value embedding of the reference data elements using a prediction neural network to generate the prediction output that characterizes the input data element.

公开(公告)号：US20230377359A1

公开(公告)日：2023-11-23

申请号：US18199129

申请日：2023-05-18

Applicant: Google LLC

Inventor： Sayna Ebrahimi ,  Sercan Omer Arik ,  Tomas Pfister

IPC: G06V30/19 ,  G06V10/70

CPC classification number: G06V30/1912 ,  G06V30/19147 ,  G06V10/70

Abstract: An aspect of the disclosed technology comprises a test-time adaptation (“TTA”) technique for visual document understanding (“VDU”) tasks that uses self-supervised learning on different modalities (e.g., text and layout) by applying masked visual language modeling (“MVLM”) along with pseudo-labeling. In accordance with an aspect of the disclosed technology, the TTA technique enables a document model to adapt to domain or distribution shifts that are detected.