公开(公告)号：US10146768B2

公开(公告)日：2018-12-04

申请号：US15415506

申请日：2017-01-25

Applicant: Google LLC

Inventor： Ariel Fuxman ,  Vivek Ramavajjala ,  Ning Ye ,  John Patrick McGregor, Jr.

IPC: G06K9/46 ,  G06F17/27 ,  G06K9/48

Abstract: Implementations relate to automatic response suggestions to images included in received messages. In some implementations, a computer-implemented method includes detecting an image posted within a first message by a first user, and programmatically analyzing the image to determine a feature vector representative of the image. The method programmatically generates one or more suggested responses to the first message based on the feature vector, each suggested response being a conversational reply to the first message. Generating the suggested responses includes determining probabilities associated with word sequences for the feature vector using a model trained with previous responses to previous images, and selecting one or more of the word sequences based on the associated probabilities. The suggested responses are determined based on the selected word sequences. The method causes the suggested responses to be rendered in the messaging application as one or more suggestions to a second user.

公开(公告)号：US20200250538A1

公开(公告)日：2020-08-06

申请号：US16265811

申请日：2019-02-01

Applicant: Google LLC

Inventor： Zhen Li ,  Yi-ting Chen ,  Ning Ye ,  Yaxi Gao ,  Zijian Guo ,  Aleksei Timofeev ,  Futang Peng ,  Thomas J. Duerig

IPC: G06N3/08 ,  G06K9/62 ,  G06F16/953 ,  G06F16/242 ,  G06N20/00 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for jointly training an image embedding model and a text embedding model. In one aspect, a method comprises: processing data from a historical query log of a search system to generate a candidate set of training examples, wherein each training example comprises: (i) a search query comprising a sequence of one or more words, (ii) an image, and (iii) selection data characterizing how often users selected the image in response to the image being identified by a search result for the search query; selecting a plurality of training examples from the candidate set of training examples; and using the training data to jointly train the image embedding model and the text embedding model.

公开(公告)号：US20240078258A1

公开(公告)日：2024-03-07

申请号：US18505776

申请日：2023-11-09

Applicant: Google LLC

Inventor： Zhen Li ,  Yi-ting Chen ,  Ning Ye ,  Yaxi Gao ,  Zijian Guo ,  Aleksei Timofeev ,  Futang Peng ,  Thomas J. Duerig

IPC: G06F16/55 ,  G06F16/242 ,  G06F16/953 ,  G06F18/22 ,  G06N3/044 ,  G06N3/084 ,  G06N20/00

CPC classification number: G06F16/55 ,  G06F16/2425 ,  G06F16/953 ,  G06F18/22 ,  G06N3/044 ,  G06N3/084 ,  G06N20/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for jointly training an image embedding model and a text embedding model. In one aspect, a method comprises: processing data from a historical query log of a search system to generate a candidate set of training examples, wherein each training example comprises: (i) a search query comprising a sequence of one or more words, (ii) an image, and (iii) selection data characterizing how often users selected the image in response to the image being identified by a search result for the search query; selecting a plurality of training examples from the candidate set of training examples; and using the training data to jointly train the image embedding model and the text embedding model.