公开(公告)号：US11893993B2

公开(公告)日：2024-02-06

申请号：US18070322

申请日：2022-11-28

Applicant: GOOGLE LLC

Inventor： Quazi Hussain ,  Adam Coimbra ,  Ilya Firman

IPC: G10L15/22 ,  G06F9/451 ,  G06F9/54 ,  G10L15/18 ,  G06F3/16 ,  G10L15/06 ,  G10L15/08 ,  G06N20/00

CPC classification number: G10L15/22 ,  G06F3/167 ,  G06F9/453 ,  G06F9/54 ,  G10L15/063 ,  G10L15/083 ,  G10L15/1815 ,  G10L15/1822 ,  G06N20/00 ,  G10L2015/0635 ,  G10L2015/223

Abstract: Dynamic interfacing with applications is provided. For example, a system receives a first input audio signal. The system processes, via a natural language processing technique, the first input audio signal to identify an application. The system activates the application for execution on the client computing device. The application declares a function the application is configured to perform. The system modifies the natural language processing technique responsive to the function declared by the application. The system receives a second input audio signal. The system processes, via the modified natural language processing technique, the second input audio signal to detect one or more parameters. The system determines that the one or more parameters are compatible for input into an input field of the application. The system generates an action data structure for the application. The system inputs the action data structure into the application, which executes the action data structure.

公开(公告)号：US11829435B2

公开(公告)日：2023-11-28

申请号：US17837914

申请日：2022-06-10

Applicant: Google LLC

Inventor： Adam Coimbra ,  Nikhil Rao ,  Hao Wu ,  Ian Baker ,  Anshul Gupta ,  Jay Akkad

IPC: G06F21/00 ,  G06F16/957 ,  G06F16/954 ,  G06F3/16 ,  G06F9/451 ,  G06F16/958 ,  G06F9/445 ,  G10L15/30 ,  H04L9/40 ,  G06F9/46 ,  G06F9/54 ,  G06F16/903 ,  G06F21/44

CPC classification number: G06F16/957 ,  G06F3/167 ,  G06F9/44589 ,  G06F9/453 ,  G06F9/468 ,  G06F9/54 ,  G06F16/90335 ,  G06F16/954 ,  G06F16/986 ,  G06F21/44 ,  G10L15/30 ,  H04L63/0421

Abstract: Secure digital assistant integration with web pages is provided. The system receives an intent manifest data structure that maps actions of a digital assistant with link templates of an electronic resource developed by a third-party developer device. The system validates the electronic resource based on the intent manifest data structure. The system receives, from a data exchange component of an iframe of the electronic resource loaded by a client computing device, an identifier of the client computing device. The system receives a foreground state of the electronic resource from an onsite state sharing API. The system selects a data value for a parameter based on the foreground state and the intent manifest data structure. The system provides the data value. An authorization component generates an authorization prompt, receives input, and transmits the data value to an onsite intent execution API of the electronic resource to execute an action.

公开(公告)号：US20230267928A1

公开(公告)日：2023-08-24

申请号：US18004971

申请日：2021-05-18

Applicant: Google LLC

Inventor： Quazi Hussain ,  Mubaraq Mishra ,  Vidhya Bhat ,  Adam Cohen ,  Sunny Goyal ,  Adam Coimbra ,  Sanjana Patel ,  Ilya Firman ,  Mehdi Alizadeh ,  Saurabh Kumar

IPC: G10L15/22 ,  G06F21/60 ,  G06F16/901 ,  H04L9/32 ,  G10L15/18

CPC classification number: G10L15/22 ,  G06F16/901 ,  G06F21/602 ,  G10L15/1822 ,  H04L9/3213 ,  G10L2015/223

Abstract: Indexing application actions for voice-based execution is provided. An indexer of a device receives, from an application executed by the device, an indication of an action and a deep link corresponding to the action declared by the application via an application programming interface. The indexer accesses an index stored in the memory of the device that is accessible to a plurality of applications. The indexer inputs, into a location in the index, the action and the deep link with a cryptographic token that prevents access to the deep link by unauthorized applications. A digital assistant of the device receives a voice query detected by a microphone and parses the voice query to determine the action. The digital assistant identifies in the index the deep link corresponding to the action. The digital assistant invokes, with the cryptographic token, the deep link to cause the application to perform the action.

公开(公告)号：US11735182B2

公开(公告)日：2023-08-22

申请号：US17192230

申请日：2021-03-04

Applicant: Google LLC

Inventor： Ulas Kirazci ,  Adam Coimbra ,  Abraham Lee ,  Wei Dong ,  Thushan Amarasiriwardena

IPC: G10L15/00 ,  G10L15/22 ,  G06F9/448 ,  G06F3/16 ,  G10L13/027

CPC classification number: G10L15/22 ,  G06F3/167 ,  G06F9/4498 ,  G10L13/027 ,  G10L2015/223 ,  G10L2015/228

Abstract: Techniques are described herein for multi-modal interaction between users, automated assistants, and other computing services. In various implementations, a user may engage with the automated assistant in order to further engage with a third party computing service. In some implementations, the user may advance through dialog state machines associated with third party computing service using both verbal input modalities and input modalities other than verbal modalities, such as visual/tactile modalities.

公开(公告)号：US20210326430A1

公开(公告)日：2021-10-21

申请号：US16466213

申请日：2019-05-06

Applicant: Google LLC

Inventor： Adam Coimbra ,  Nikhil Rao ,  Hao Wu ,  Ian Baker ,  Anshul Gupta ,  Jay Akkad

IPC: G06F21/44 ,  G06F16/903

Abstract: Secure digital assistant integration with web pages is provided. The system receives an intent manifest data structure that maps actions of a digital assistant with link templates of an electronic resource developed by a third-party developer device. The system validates the electronic resource based on the intent manifest data structure. The system receives, from a data exchange component of an iframe of the electronic resource loaded by a client computing device, an identifier of the client computing device. The system receives a foreground state of the electronic resource from an onsite state sharing API. The system selects a data value for a parameter based on the foreground state and the intent manifest data structure. The system provides the data value. An authorization component generates an authorization prompt, receives input, and transmits the data value to an onsite intent execution API of the electronic resource to execute an action.

公开(公告)号：US11145300B2

公开(公告)日：2021-10-12

申请号：US16064961

申请日：2018-05-07

Applicant: Google LLC

Inventor： Gaurav Bhaya ,  Ulas Kirazci ,  Bradley Abrams ,  Adam Coimbra ,  Ilya Firman ,  Carey Radebaugh

IPC: G10L15/22 ,  G10L15/18 ,  G10L15/30 ,  G06F40/216 ,  G06F40/30 ,  G06F16/901 ,  G06F40/35

Abstract: The present disclosure is generally directed to the generation of voice-activated data flows in interconnected network. The voice-activated data flows can include input audio signals that include a request and are detected at a client device. The client device can transmit the input audio signal to a data processing system, where the input audio signal can be parsed and passed to the data processing system of a service provider to fulfill the request in the input audio signal. The present solution is configured to conserve network resources by reducing the number of network transmissions needed to fulfill a request.

公开(公告)号：US20210193146A1

公开(公告)日：2021-06-24

申请号：US17192230

申请日：2021-03-04

Applicant: Google LLC

Inventor： Ulas Kirazci ,  Adam Coimbra ,  Abraham Lee ,  Wei Dong ,  Thushan Amarasiriwardena

IPC: G10L15/22 ,  G06F9/448 ,  G06F3/16 ,  G10L13/027

Abstract: Techniques are described herein for multi-modal interaction between users, automated assistants, and other computing services. In various implementations, a user may engage with the automated assistant in order to further engage with a third party computing service. In some implementations, the user may advance through dialog state machines associated with third party computing service using both verbal input modalities and input modalities other than verbal modalities, such as visual/tactile modalities.

公开(公告)号：US20200294497A1

公开(公告)日：2020-09-17

申请号：US15774950

申请日：2018-05-07

Applicant: Google LLC

Inventor： Ulas Kirazci ,  Adam Coimbra ,  Abraham Lee ,  Wei Dong ,  Thushan Amarasiriwardena

IPC: G10L15/22 ,  G10L13/027 ,  G06F3/16 ,  G06F9/448

Abstract: Techniques are described herein for multi-modal interaction between users, automated assistants, and other computing services. In various implementations, a user may engage with the automated assistant in order to further engage with a third party computing service. In some implementations, the user may advance through dialog state machines associated with third party computing service using both verbal input modalities and input modalities other than verbal modalities, such as visual/tactile modalities.