公开(公告)号：US20220245428A1

公开(公告)日：2022-08-04

申请号：US17592796

申请日：2022-02-04

Applicant: Google LLC

Inventor： Yi Tay ,  Da-Cheng Juan ,  Dara Bahri ,  Donald Arthur Metzler, JR. ,  Jai Prakash Gupta ,  Mostafa Dehghani ,  Phillip Pham ,  Vamsi Krishna Aribandi ,  Zhen Qin

IPC: G06N3/04 ,  G06N3/10

Abstract: Provided are machine-learned attention models that feature omnidirectional processing, example implementations of which can be referred to as Omnidirectional Representations from Transformers (OMNINET). In example models described in the present disclosure, instead of maintaining a strictly horizontal receptive field, each token is allowed to attend to all tokens in some or all of the other tokens across the entire network.

公开(公告)号：US20230267277A1

公开(公告)日：2023-08-24

申请号：US18010727

申请日：2020-06-15

Applicant: Google LLC

Inventor： Weize Kong ,  Michael Bendersky ,  Marc Najork ,  Rama Kumar Pasumarthi ,  Zhen Qin ,  Rolf Jagerman

IPC: G06F40/30 ,  G06N20/00 ,  G06F16/9538 ,  G06F16/9535

CPC classification number: G06F40/30 ,  G06N20/00 ,  G06F16/9538 ,  G06F16/9535

Abstract: Systems and methods of the present disclosure are directed to a method for training a machine-learned semantic matching model. The method can include obtaining a first and second document and a first and second activity log. The method can include determining, based on the first document activity log and the second document activity log, a relation label indicative of whether the documents are related. The method can include inputting the documents into the model to receive a semantic similarity value representing an estimated semantic similarity between the first document and the second document. The method can include evaluating a loss function that evaluates a difference between the relation label and the semantic similarity value. The method can include modifying values of parameters of the model based on the loss function.

公开(公告)号：US20250124067A1

公开(公告)日：2025-04-17

申请号：US18913702

申请日：2024-10-11

Applicant: Google LLC

Inventor： Zhen Qin ,  Rolf Jagerman ,  Kai Hui ,  Honglei Zhuang ,  Junru Wu ,  Jiaming Shen ,  Tianqi Liu ,  Jialu Liu ,  Donald Arthur Metzler, JR. ,  Xuanhui Wang ,  Michael Bendersky

IPC: G06F16/338

Abstract: Provided are computing systems, methods, and platforms that rank text with pairwise ranking prompting using a generative sequence processing model. A prompt comprising a query and sets of text associated with candidate results can be generated. The generative sequence processing model can be prompted with the prompt and perform pairwise comparisons between the sets of text in the prompt based on the query in the prompt. An output can be generated that ranks the sets of text in response to the query.

公开(公告)号：US20240289552A1

公开(公告)日：2024-08-29

申请号：US18564859

申请日：2022-05-27

Applicant: Google LLC

Inventor： Yi Tay ,  Dara Bahri ,  Donald Arthur Metzler, Jr. ,  Hyung Won Chung ,  Jai Prakash Gupta ,  Sebastian Nikolas Ruder ,  Simon Baumgartner ,  Vinh Quoc Tran ,  Zhen Qin

IPC: G06F40/284

CPC classification number: G06F40/284

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing a machine learning task on an input sequence of characters that has a respective character at each of a plurality of character positions to generate a network output. One of the systems includes a neural network configured to perform the machine learning task, the neural network comprising a gradient-based sub-word tokenizer and an output neural network. The gradient-based sub-word tokenizer is configured to apply a learned, i.e., flexible, sub-word tokenization strategy to the input sequence of characters to generate a sequence of latent sub-word representations. The output neural network is configured to process the latent sub-word representation to generate the network output for the task.