公开(公告)号：US11688191B2

公开(公告)日：2023-06-27

申请号：US17941971

申请日：2022-09-09

Applicant: GOOGLE LLC

Inventor： Ibrahim Badr ,  Nils Grimsmo ,  Gokhan H. Bakir ,  Kamil Anikiej ,  Aayush Kumar ,  Viacheslav Kuznetsov

IPC: G06V30/262 ,  G06F16/58 ,  G06F16/9032 ,  G06V20/62 ,  G06V10/70 ,  G06V30/10

CPC classification number: G06V30/262 ,  G06F16/5866 ,  G06F16/9032 ,  G06V10/768 ,  G06V20/63 ,  G06V30/10

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for contextually disambiguating queries are disclosed. In an aspect, a method includes receiving an image being presented on a display of a computing device and a transcription of an utterance spoken by a user of the computing device, identifying a particular sub-image that is included in the image, and based on performing image recognition on the particular sub-image, determining one or more first labels that indicate a context of the particular sub-image. The method also includes, based on performing text recognition on a portion of the image other than the particular sub-image, determining one or more second labels that indicate the context of the particular sub-image, based on the transcription, the first labels, and the second labels, generating a search query, and providing, for output, the search query.

公开(公告)号：US20230004597A1

公开(公告)日：2023-01-05

申请号：US17941971

申请日：2022-09-09

Applicant: GOOGLE LLC

Inventor： Ibrahim Badr ,  Nils Grimsmo ,  Gokhan H. Bakir ,  Kamil Anikiej ,  Aayush Kumar ,  Viacheslav Kuznetsov

IPC: G06F16/58 ,  G06F16/9032 ,  G06V10/70 ,  G06V20/62

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for contextually disambiguating queries are disclosed. In an aspect, a method includes receiving an image being presented on a display of a computing device and a transcription of an utterance spoken by a user of the computing device, identifying a particular sub-image that is included in the image, and based on performing image recognition on the particular sub-image, determining one or more first labels that indicate a context of the particular sub-image. The method also includes, based on performing text recognition on a portion of the image other than the particular sub-image, determining one or more second labels that indicate the context of the particular sub-image, based on the transcription, the first labels, and the second labels, generating a search query, and providing, for output, the search query.

公开(公告)号：US11442983B2

公开(公告)日：2022-09-13

申请号：US16731786

申请日：2019-12-31

Applicant: Google LLC

Inventor： Ibrahim Badr ,  Nils Grimsmo ,  Gokhan H. Bakir ,  Kamil Anikiej ,  Aayush Kumar ,  Viacheslav Kuznetsov

IPC: G06F16/58 ,  G06F16/9032 ,  G06V10/70 ,  G06V20/62 ,  G06V30/10

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for contextually disambiguating queries are disclosed. In an aspect, a method includes receiving an image being presented on a display of a computing device and a transcription of an utterance spoken by a user of the computing device, identifying a particular sub-image that is included in the image, and based on performing image recognition on the particular sub-image, determining one or more first labels that indicate a context of the particular sub-image. The method also includes, based on performing text recognition on a portion of the image other than the particular sub-image, determining one or more second labels that indicate the context of the particular sub-image, based on the transcription, the first labels, and the second labels, generating a search query, and providing, for output, the search query.

公开(公告)号：US20200250227A1

公开(公告)日：2020-08-06

申请号：US16731786

申请日：2019-12-31

Applicant: Google LLC

Inventor： Ibrahim Badr ,  Nils Grimsmo ,  Gokhan H. Bakir ,  Kamil Anikiej ,  Aayush Kumar ,  Viacheslav Kuznetsov

IPC: G06F16/58 ,  G06K9/32 ,  G06F16/9032 ,  G06K9/72

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for contextually disambiguating queries are disclosed. In an aspect, a method includes receiving an image being presented on a display of a computing device and a transcription of an utterance spoken by a user of the computing device, identifying a particular sub-image that is included in the image, and based on performing image recognition on the particular sub-image, determining one or more first labels that indicate a context of the particular sub-image. The method also includes, based on performing text recognition on a portion of the image other than the particular sub-image, determining one or more second labels that indicate the context of the particular sub-image, based on the transcription, the first labels, and the second labels, generating a search query, and providing, for output, the search query.

公开(公告)号：US20250054405A1

公开(公告)日：2025-02-13

申请号：US18446125

申请日：2023-08-08

Applicant: Google LLC

Inventor： Jessica Lee ,  Kimiya Hojjat ,  David Trotter Oleson ,  Daniel Valcarce Silva ,  Andrea D'olimpio ,  Urs Christian Lukas Dönni ,  Christopher Rohrs ,  Kuba Dolecki ,  Balint Miklos ,  Federico Chialvo ,  Lisa Wang ,  Jieru Hu ,  Ryan Muller ,  Chris Heather ,  Sara Wiltberger ,  Saurabh Paliwal ,  Viacheslav Kuznetsov ,  Gleb Makarchuk ,  Philipp Neubeck ,  Ivan Jurin

IPC: G09B7/02 ,  G06F16/9535 ,  G06F16/9538 ,  G06F40/40

Abstract: The present disclosure provides computer-implemented methods, systems, and devices for generating multistep explanations for pedagogical exercises. A computing device receives a query from a user. The computing device determines that the query includes query data describing a pedagogical exercise to be solved. The computing device provides the query data as input to an explanatory machine-learned model. The computing device receives, as output from the explanatory machine-learned model, a pedagogical response, the pedagogical response including a multi-step explanation of a solution to the pedagogical exercise. The computing device provides the pedagogical response for display to a user.

公开(公告)号：US10565256B2

公开(公告)日：2020-02-18

申请号：US15463018

申请日：2017-03-20

Applicant: Google LLC

Inventor： Ibrahim Badr ,  Nils Grimsmo ,  Gokhan H. Bakir ,  Kamil Anikiej ,  Aayush Kumar ,  Viacheslav Kuznetsov

IPC: G06F16/58 ,  G06K9/72 ,  G06F16/9032 ,  G06K9/32

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for contextually disambiguating queries are disclosed. In an aspect, a method includes receiving an image being presented on a display of a computing device and a transcription of an utterance spoken by a user of the computing device, identifying a particular sub-image that is included in the image, and based on performing image recognition on the particular sub-image, determining one or more first labels that indicate a context of the particular sub-image. The method also includes, based on performing text recognition on a portion of the image other than the particular sub-image, determining one or more second labels that indicate the context of the particular sub-image, based on the transcription, the first labels, and the second labels, generating a search query, and providing, for output, the search query.