公开(公告)号：WO2018174849A1

公开(公告)日：2018-09-27

申请号：PCT/US2017/023217

申请日：2017-03-20

Applicant: GOOGLE LLC

Inventor： BADR, Ibrahim ,  GRIMSMO, Nils ,  BAKIR, Gokhan H. ,  ANIKIEJ, Kamil ,  KUMAR, Aayush ,  KUZNETSOV, Viacheslav

IPC: G06F17/30 ,  G06K9/00

CPC classification number: G06F16/951 ,  G06F16/434 ,  G06F16/9032 ,  G06K9/00671 ,  G06K9/72 ,  G10L15/26

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for contextually disambiguating queries are disclosed. In an aspect, a method includes receiving an image being presented on a display of a computing device and a transcription of an utterance spoken by a user of the computing device, identifying a particular sub-image that is included in the image, and based on performing image recognition on the particular sub-image, determining one or more first labels that indicate a context of the particular sub-image. The method also includes, based on performing text recognition on a portion of the image other than the particular sub-image, determining one or more second labels that indicate the context of the particular sub-image, based on the transcription, the first labels, and the second labels, generating a search query, and providing, for output, the search query.

公开(公告)号：WO2020005575A1

公开(公告)日：2020-01-02

申请号：PCT/US2019/037171

申请日：2019-06-14

Applicant: GOOGLE LLC

Inventor： BADR, Ibrahim ,  BAKIR, Gokhan H. ,  KEHL, Roland Peter ,  GRIMSMO, Nils

IPC: H04N21/436 ,  G06K9/00 ,  H04N1/00 ,  G06F3/03

Abstract: Methods, systems, and apparatus for controlling smart devices are described. In one aspect a method includes receiving image data for an image captured by a camera of a mobile device of a user and determining that the image depicts at least one of a smart device or a physical control for the smart device. In response to determining that that the image depicts a smart device or a physical control for the smart device, identifying one or more user interface controls for controlling the smart device, and generating and presenting, at a display of the mobile device, the one or more user interface controls for controlling the smart device. The method can further include detecting, at the display of the mobile device, user interaction with at least one of the one or more user interface controls, and controlling the smart device based on the detected user interaction.

公开(公告)号：WO2018213321A1

公开(公告)日：2018-11-22

申请号：PCT/US2018/032786

申请日：2018-05-15

Applicant: GOOGLE LLC

Inventor： BADR, Ibrahim ,  GRIMSMO, Nils ,  BAKIR, Gökhan

IPC: G06F3/16 ,  G06F17/30

Abstract: Methods, apparatus, and computer readable media are described related to causing processing of sensor data to be performed in response to determining a request related to an environmental object that is likely captured by the sensor data. Some implementations further relate to determining whether the request is resolvable based on the processing of the sensor data. When it is determined that the request is not resolvable, a prompt is determined and provided as user interface output, where the prompt provides guidance on further input that will enable the request to be resolved. In those implementations, the further input ( e.g. , additional sensor data and/or the user interface input) received in response to the prompt can then be utilized to resolve the request.