公开(公告)号：US20250068847A1

公开(公告)日：2025-02-27

申请号：US18453236

申请日：2023-08-21

Applicant: Google LLC

Inventor： Vincent Perot ,  Florian Luisier ,  Kai Kang ,  Ramya Sree Boppana ,  Jiaqi Mu ,  Xiaoyu Sun ,  Carl Elie Saroufim ,  Guolong Su ,  Hao Zhang ,  Nikolay Alexeevich Glushnev ,  Nan Hua ,  Yun-Hsuan Sung ,  Michael Yiupun Kwong

IPC: G06F40/295 ,  G06V30/19

Abstract: Systems and methods for performing document entity extraction are described herein. The method can include receiving an inference document and a target schema. The method can also include generating one or more document inputs from the inference document and one or more schema inputs from the target schema. The method can further include, for each combination of the document input and schema input, obtaining one or more extraction inputs by generating a respective extraction input based on the combination, providing the respective extraction input to the machine-learned model, and receiving a respective output of the machine-learned model based on the respective extraction. The method can also include validating the extracted entity data based on reference spatial locations and inference spatial locations and outputting the validated extracted entity data.

公开(公告)号：US20240273294A1

公开(公告)日：2024-08-15

申请号：US18166806

申请日：2023-02-09

Applicant: Google LLC

Inventor： Siamak Shakeri ,  Cicero Nogueira dos Santos ,  Daniel Matthew Cer ,  Zhe Dong ,  Jianmo Ni ,  Yun-Hsuan Sung ,  John Nham

IPC: G06F40/295 ,  G06N3/0455 ,  G06N3/084

CPC classification number: G06F40/295 ,  G06N3/0455 ,  G06N3/084

Abstract: The technology employs soft knowledge prompts (KPs) to inject relevant world knowledge into language models. This includes training KPs via self-supervised learning on data from one or more knowledge bases. KPs are task independent and can function as an external memory of the language models. KPs may be entity-centric, meaning that each prompt primarily encodes information about one entity from a given knowledge base. A method includes identifying a KP in response to a received input text, concatenating that KP to a sequence of word embeddings of the input text, applying the concatenated information to a trained language model, predicting an object entity name, computing a cross-entropy loss, and updating the identified KP based on the computed cross-entropy loss.

公开(公告)号：US20240062111A1

公开(公告)日：2024-02-22

申请号：US18386015

申请日：2023-11-01

Applicant: GOOGLE LLC

Inventor： Brian Strope ,  Yun-Hsuan Sung ,  Wangqing Yuan

IPC: G06N20/00 ,  G06F16/33 ,  G06F16/332 ,  G06F16/35 ,  G06N5/04

CPC classification number: G06N20/00 ,  G06F16/3329 ,  G06F16/3344 ,  G06F16/3346 ,  G06F16/35 ,  G06N5/04

Abstract: Systems, methods, and computer readable media related to: training an encoder model that can be utilized to determine semantic similarity of a natural language textual string to each of one or more additional natural language textual strings (directly and/or indirectly); and/or using a trained encoder model to determine one or more responsive actions to perform in response to a natural language query. The encoder model is a machine learning model, such as a neural network model. In some implementations of training the encoder model, the encoder model is trained as part of a larger network architecture trained based on one or more tasks that are distinct from a “semantic textual similarity” task for which the encoder model can be used.

公开(公告)号：US20250068913A1

公开(公告)日：2025-02-27

申请号：US18828690

申请日：2024-09-09

Applicant: GOOGLE LLC

Inventor： Brian Strope ,  Yun-Hsuan Sung ,  Matthew Henderson ,  Rami Al-Rfou' ,  Raymond Kurzweil

IPC: G06N3/084 ,  G06F16/00 ,  G06F16/335 ,  G06N3/044 ,  G06N3/045 ,  G06N5/04

Abstract: Systems, methods, and computer readable media related to information retrieval. Some implementations are related to training and/or using a relevance model for information retrieval. The relevance model includes an input neural network model and a subsequent content neural network model. The input neural network model and the subsequent content neural network model can be separate, but trained and/or used cooperatively. The input neural network model and the subsequent content neural network model can be “separate” in that separate inputs are applied to the neural network models, and each of the neural network models is used to generate its own feature vector based on its applied input. A comparison of the feature vectors generated based on the separate network models can then be performed, where the comparison indicates relevance of the input applied to the input neural network model to the separate input applied to the subsequent content neural network model.

公开(公告)号：US20240370487A1

公开(公告)日：2024-11-07

申请号：US18253859

申请日：2022-11-04

Applicant: Google LLC

Inventor： Severin Heiniger ,  Balint Miklos ,  Yun-Hsuan Sung ,  Zhen Li ,  Yinfei Yang ,  Chao Jia

IPC: G06F16/538 ,  G06F16/55 ,  G06N3/084

Abstract: Systems and methods of the present disclosure are directed to computer-implemented method for machine-learned multimodal search refinement. The method includes obtaining a query image embedding for a query image and a textual query refinement associated with the query image. The method includes processing the query image embedding and the textual query refinement with a machine-learned query refinement model to obtain a refined query image embedding that incorporates the textual query refinement. The method includes evaluating a loss function that evaluates a distance between the refined query image embedding and an embedding for a ground truth image within an image embedding space. The method includes modifying value(s) of parameter(s) of the machine-learned query refinement model based on the loss function.