公开(公告)号：US12038970B2

公开(公告)日：2024-07-16

申请号：US18171511

申请日：2023-02-20

Applicant: Google LLC

Inventor： Zhen Li ,  Yi-Ting Chen ,  Yaxi Gao ,  Da-Cheng Juan ,  Aleksei Timofeev ,  Chun-Ta Lu ,  Futang Peng ,  Sujith Ravi ,  Andrew Tomkins ,  Thomas J. Duerig

IPC: G06F16/00 ,  G06F16/538 ,  G06F16/55 ,  G06F16/9538 ,  G06F18/214 ,  G06F18/22 ,  G06F18/40 ,  G06N3/042 ,  G06N3/044 ,  G06N3/084

CPC classification number: G06F16/55 ,  G06F16/538 ,  G06F16/9538 ,  G06F18/2148 ,  G06F18/22 ,  G06F18/41 ,  G06N3/042 ,  G06N3/044 ,  G06N3/084

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an image embedding model. In one aspect, a method comprises: obtaining training data comprising a plurality of training examples, wherein each training example comprises: an image pair comprising a first image and a second image; and selection data indicating one or more of: (i) a co-click rate of the image pair, and (ii) a similar-image click rate of the image pair; and using the training data to train an image embedding model having a plurality of image embedding model parameters.

公开(公告)号：US11586927B2

公开(公告)日：2023-02-21

申请号：US16265793

申请日：2019-02-01

Applicant: Google LLC

Inventor： Zhen Li ,  Yi-ting Chen ,  Yaxi Gao ,  Da-Cheng Juan ,  Aleksei Timofeev ,  Chun-Ta Lu ,  Futang Peng ,  Sujith Ravi ,  Andrew Tomkins ,  Thomas J. Duerig

IPC: G06K9/00 ,  G06N3/084 ,  G06F16/538 ,  G06F16/9538 ,  G06K9/62 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an image embedding model. In one aspect, a method comprises: obtaining training data comprising a plurality of training examples, wherein each training example comprises: an image pair comprising a first image and a second image; and selection data indicating one or more of: (i) a co-click rate of the image pair, and (ii) a similar-image click rate of the image pair; and using the training data to train an image embedding model having a plurality of image embedding model parameters.

公开(公告)号：US20220245428A1

公开(公告)日：2022-08-04

申请号：US17592796

申请日：2022-02-04

Applicant: Google LLC

Inventor： Yi Tay ,  Da-Cheng Juan ,  Dara Bahri ,  Donald Arthur Metzler, JR. ,  Jai Prakash Gupta ,  Mostafa Dehghani ,  Phillip Pham ,  Vamsi Krishna Aribandi ,  Zhen Qin

IPC: G06N3/04 ,  G06N3/10

Abstract: Provided are machine-learned attention models that feature omnidirectional processing, example implementations of which can be referred to as Omnidirectional Representations from Transformers (OMNINET). In example models described in the present disclosure, instead of maintaining a strictly horizontal receptive field, each token is allowed to attend to all tokens in some or all of the other tokens across the entire network.

公开(公告)号：US20200250537A1

公开(公告)日：2020-08-06

申请号：US16265793

申请日：2019-02-01

Applicant: Google LLC

Inventor： Zhen Li ,  Yi-ting Chen ,  Yaxi Gao ,  Da-Cheng Juan ,  Aleksei Timofeev ,  Chun-Ta Lu ,  Futang Peng ,  Sujith Ravi ,  Andrew Tomkins ,  Thomas J. Duerig

IPC: G06N3/08 ,  G06K9/62 ,  G06F16/9538 ,  G06F16/538 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an image embedding model. In one aspect, a method comprises: obtaining training data comprising a plurality of training examples, wherein each training example comprises: an image pair comprising a first image and a second image; and selection data indicating one or more of: (i) a co-click rate of the image pair, and (ii) a similar-image click rate of the image pair; and using the training data to train an image embedding model having a plurality of image embedding model parameters.

公开(公告)号：US20240330361A1

公开(公告)日：2024-10-03

申请号：US18741082

申请日：2024-06-12

Applicant: Google LLC

Inventor： Zhen Li ,  Yi-Ting Chen ,  Yaxi Gao ,  Da-Cheng Juan ,  Aleksei Timofeev ,  Chun-Ta Lu ,  Futang Peng ,  Sujith Ravi ,  Andrew Tomkins ,  Thomas J. Duerig

IPC: G06F16/55 ,  G06F16/538 ,  G06F16/9538 ,  G06F18/214 ,  G06F18/22 ,  G06F18/40 ,  G06N3/042 ,  G06N3/044 ,  G06N3/084

CPC classification number: G06F16/55 ,  G06F16/538 ,  G06F16/9538 ,  G06F18/2148 ,  G06F18/22 ,  G06F18/41 ,  G06N3/042 ,  G06N3/044 ,  G06N3/084

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an image embedding model. In one aspect, a method comprises: obtaining training data comprising a plurality of training examples, wherein each training example comprises: an image pair comprising a first image and a second image; and selection data indicating one or more of: (i) a co-click rate of the image pair, and (ii) a similar-image click rate of the image pair; and using the training data to train an image embedding model having a plurality of image embedding model parameters.

公开(公告)号：US20230205813A1

公开(公告)日：2023-06-29

申请号：US18171511

申请日：2023-02-20

Applicant: Google LLC

Inventor： Zhen Li ,  Yi-Ting Chen ,  Yaxi Gao ,  Da-Cheng Juan ,  Aleksei Timofeev ,  Chun-Ta Lu ,  Futang Peng ,  Sujith Ravi ,  Andrew Tomkins ,  Thomas J. Duerig

IPC: G06F16/55 ,  G06F16/538 ,  G06F16/9538 ,  G06N3/084 ,  G06F18/22 ,  G06F18/40 ,  G06F18/214 ,  G06N3/042 ,  G06N3/044

CPC classification number: G06F16/55 ,  G06F16/538 ,  G06F16/9538 ,  G06N3/084 ,  G06F18/22 ,  G06F18/41 ,  G06F18/2148 ,  G06N3/042 ,  G06N3/044

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an image embedding model. In one aspect, a method comprises: obtaining training data comprising a plurality of training examples, wherein each training example comprises: an image pair comprising a first image and a second image; and selection data indicating one or more of: (i) a co-click rate of the image pair, and (ii) a similar-image click rate of the image pair; and using the training data to train an image embedding model having a plurality of image embedding model parameters.

公开(公告)号：US20210248450A1

公开(公告)日：2021-08-12

申请号：US17169718

申请日：2021-02-08

Applicant: Google LLC

Inventor： Yi Tay ,  Liu Yang ,  Donald Arthur Metzler, JR. ,  Dara Bahri ,  Da-Cheng Juan

IPC: G06N3/04 ,  G06N20/00 ,  G06F17/16

Abstract: A system for performing a machine learning task on a network input is described. The system includes one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to implement (i) multiple sorting networks in which each sorting network is configured to sort vector blocks in a sequence of vector blocks to generate a sorted sequence of vector blocks; and (ii) a sorting attention neural network configured to perform the machine learning task on the input sequence by executing multiple sorting attention mechanisms using the sorting networks.