SYSTEMS, METHODS, AND APPARATUS FOR IMAGE-RESPONSIVE AUTOMATED ASSISTANTS

    公开(公告)号:US20190080169A1

    公开(公告)日:2019-03-14

    申请号:US15700106

    申请日:2017-09-09

    申请人: Google Inc.

    IPC分类号: G06K9/00 H04N5/232

    摘要: Techniques described herein enable a user to interact with an automated assistant and obtain relevant output from the automated assistant without requiring arduous typed input to be provided by the user and/or without requiring the user to provide spoken input that could cause privacy concerns (e.g., if other individuals are nearby). The assistant application can operate in multiple different image conversation modes in which the assistant application is responsive to various objects in a field of view of the camera. The image conversation modes can be suggested to the user when a particular object is detected in the field of view of the camera. When the user selects an image conversation mode, the assistant application can thereafter provide output, for presentation, that is based on the selected image conversation mode and that is based on object(s) captured by image(s) of the camera.

    Storing metadata related to captured images

    公开(公告)号:US10469755B2

    公开(公告)日:2019-11-05

    申请号:US15602661

    申请日:2017-05-23

    申请人: Google Inc.

    摘要: The present disclosure relates to user-selected metadata related to images captured by a camera of a client device. User-selected metadata may include contextual information and/or information provided by a user when the images are captured. In various implementations, a free form input may be received at a first client device of one or more client devices operated by a user. A task request may be recognized from the free form input, and it may be determined that the task request includes a request to store metadata related to one or more images captured by a camera of the first client device. The metadata may be selected based on content of the task request. The metadata may then be stored, e.g., in association with one or more images captured by the camera, in computer-readable media. The computer-readable media may be searchable by the metadata.

    Systems, methods, and apparatus for providing image shortcuts for an assistant application

    公开(公告)号:US10366291B2

    公开(公告)日:2019-07-30

    申请号:US15700104

    申请日:2017-09-09

    申请人: Google Inc.

    摘要: Methods, apparatus, systems, and computer-readable media are set forth for generating and/or utilizing image shortcuts that cause one or more corresponding computer actions to be performed in response to determining that one or more features are present in image(s) from a camera of a computing device of a user (e.g., present in a real-time image feed from the camera). An image shortcut can be generated in response to user interface input, such as a spoken command. For example, the user interface input can direct the automated assistant to perform one or more actions in response to object(s) having certain feature(s) being present in a field of view of the camera. Subsequently, when the user directs their camera at object(s) having such feature(s), the assistant application can cause the action(s) to be automatically performed. For example, the assistant application can cause data to be presented and/or can control a remote device in accordance with the image shortcut.

    OBTAINING RESPONSIVE INFORMATION FROM MULTIPLE CORPORA

    公开(公告)号:US20190005138A1

    公开(公告)日:2019-01-03

    申请号:US15703205

    申请日:2017-09-13

    申请人: Google Inc.

    摘要: Techniques are described herein for automated assistants that search various alternative corpora for information. In various implementations, a method may include receiving, by an automated assistant via an input component of a first client device, a free form input, wherein the free form input includes a request for specific information; searching a general purpose corpus of online documents to obtain a first set of candidate response(s) to the request for specific information; searching a user-specific corpus of active document(s) to obtain a second set of candidate response(s) to the request for specific information; comparing the first and second sets of candidate responses; based on the comparing, selecting a given response to the request for specific information from the first or second set; and providing, by the automated assistant, output indicative of the given response.

    RESOLVING AUTOMATED ASSISTANT REQUESTS THAT ARE BASED ON IMAGE(S) AND/OR OTHER SENSOR DATA

    公开(公告)号:US20180336414A1

    公开(公告)日:2018-11-22

    申请号:US15631274

    申请日:2017-06-23

    申请人: Google Inc.

    IPC分类号: G06K9/00 H04L12/58

    摘要: Methods, apparatus, and computer readable media are described related to causing processing of sensor data to be performed in response to determining a request related to an environmental object that is likely captured by the sensor data. Some implementations further relate to determining whether the request is resolvable based on the processing of the sensor data. When it is determined that the request is not resolvable, a prompt is determined and provided as user interface output, where the prompt provides guidance on further input that will enable the request to be resolved. In those implementations, the further input (e.g., additional sensor data and/or the user interface input) received in response to the prompt can then be utilized to resolve the request.

    Obtaining responsive information from multiple corpora

    公开(公告)号:US11017037B2

    公开(公告)日:2021-05-25

    申请号:US15703205

    申请日:2017-09-13

    申请人: Google Inc.

    摘要: Techniques are described herein for automated assistants that search various alternative corpora for information. In various implementations, a method may include receiving, by an automated assistant via an input component of a first client device, a free form input, wherein the free form input includes a request for specific information; searching a general purpose corpus of online documents to obtain a first set of candidate response(s) to the request for specific information; searching a user-specific corpus of active document(s) to obtain a second set of candidate response(s) to the request for specific information; comparing the first and second sets of candidate responses; based on the comparing, selecting a given response to the request for specific information from the first or second set; and providing, by the automated assistant, output indicative of the given response.

    Systems, methods, and apparatus for image-responsive automated assistants

    公开(公告)号:US10607082B2

    公开(公告)日:2020-03-31

    申请号:US15700106

    申请日:2017-09-09

    申请人: Google Inc.

    摘要: Techniques described herein enable a user to interact with an automated assistant and obtain relevant output from the automated assistant without requiring arduous typed input to be provided by the user and/or without requiring the user to provide spoken input that could cause privacy concerns (e.g., if other individuals are nearby). The assistant application can operate in multiple different image conversation modes in which the assistant application is responsive to various objects in a field of view of the camera. The image conversation modes can be suggested to the user when a particular object is detected in the field of view of the camera. When the user selects an image conversation mode, the assistant application can thereafter provide output, for presentation, that is based on the selected image conversation mode and that is based on object(s) captured by image(s) of the camera.

    Providing contextual actions for mobile onscreen content

    公开(公告)号:US10535005B1

    公开(公告)日:2020-01-14

    申请号:US15386771

    申请日:2016-12-21

    申请人: GOOGLE INC.

    摘要: Systems and methods provide an application programming interface to offer action suggestions to third-party applications using context data associated with the third-party. An example method includes receiving content information and context information from a source mobile application, the content information representing information to be displayed on a mobile device as part of a source mobile application administered by a third party, the context information being information specific to the third party and unavailable to a screen scraper. The method also includes predicting an action based on the content information and the context information, the action representing a deep link for a target mobile application. The method further includes providing the action to the source mobile application with a title and a thumbnail, the source mobile application using the title and thumbnail to display a selectable control that, when selected, causes the mobile device to initiate the action.

    Resolving automated assistant requests that are based on image(s) and/or other sensor data

    公开(公告)号:US10275651B2

    公开(公告)日:2019-04-30

    申请号:US15631274

    申请日:2017-06-23

    申请人: Google Inc.

    IPC分类号: G06K9/00 H04L12/58

    摘要: Methods, apparatus, and computer readable media are described related to causing processing of sensor data to be performed in response to determining a request related to an environmental object that is likely captured by the sensor data. Some implementations further relate to determining whether the request is resolvable based on the processing of the sensor data. When it is determined that the request is not resolvable, a prompt is determined and provided as user interface output, where the prompt provides guidance on further input that will enable the request to be resolved. In those implementations, the further input (e.g., additional sensor data and/or the user interface input) received in response to the prompt can then be utilized to resolve the request.