公开(公告)号：US11086959B2

公开(公告)日：2021-08-10

申请号：US16899016

申请日：2020-06-11

Applicant: Google LLC

Inventor： Adam Coimbra ,  Nikhil Rao ,  Hao Wu ,  Ian Baker ,  Anshul Gupta ,  Jay Akkad

IPC: G06F9/44 ,  G06F16/957 ,  G06F16/954 ,  G06F3/16 ,  G06F9/451 ,  G06F16/958 ,  G06F9/445 ,  G10L15/30 ,  H04L29/06 ,  G06F9/46 ,  G06F9/54

Abstract: Secure digital assistant integration with web pages is provided. The system receives an intent manifest data structure that maps actions of a digital assistant with link templates of an electronic resource developed by a third-party developer device. The system validates the electronic resource based on the intent manifest data structure. The system receives, from a data exchange component of an iframe of the electronic resource loaded by a client computing device, an identifier of the client computing device. The system receives a foreground state of the electronic resource from an onsite state sharing API. The system selects a data value for a parameter based on the foreground state and the intent manifest data structure. The system provides the data value. An authorization component generates an authorization prompt, receives input, and transmits the data value to an onsite intent execution API of the electronic resource to execute an action.

公开(公告)号：US20190341040A1

公开(公告)日：2019-11-07

申请号：US16269275

申请日：2019-02-06

Applicant: Google LLC

Inventor： Ulas Kirazci ,  Adam Coimbra ,  Abraham Lee ,  Wei Dong ,  Thushan Amarasiriwardena ,  Yudong Sun ,  Xiao Gao

IPC: G10L15/22 ,  G10L13/02 ,  G06F3/16 ,  G06F3/0485

Abstract: Techniques are described herein for multi-modal interaction between users, automated assistants, and other computing services. In various implementations, a user may engage with the automated assistant in order to further engage with a third party computing service. In some implementations, the user may advance through dialog state machines associated with third party computing service using both verbal input modalities and input modalities other than verbal modalities, such as visual/tactile modalities.

公开(公告)号：US20250045082A1

公开(公告)日：2025-02-06

申请号：US18793675

申请日：2024-08-02

Applicant: GOOGLE LLC

Inventor： Cliff Kuang ,  Adam Coimbra ,  Bogdan Prisacari ,  Felix Weissenberger ,  Eric Stavarache ,  Mugurel-Ionut Andreica ,  Jonathan Splitlog ,  Caleb Misclevitz

IPC: G06F9/451 ,  G06F3/0484 ,  G06V40/20

Abstract: Implementations relate to determining a general routine when an automated assistant is not configured to fulfill a user request. When the user submits a request to an automated assistant to perform a routine and the automated assistant is not configured to fulfill the request, the user demonstrates the actions that are included in the routine. The automated assistant generates a routine based on the actions of the user and stores the routine with the request that was initially submitted by the user. In some implementations, a general routine can include one or more parameters and the user provides a value for the parameters with the request. General routines can additionally be generated based on previous routines performed by the user and/or other users.

公开(公告)号：US20240274132A1

公开(公告)日：2024-08-15

申请号：US18642010

申请日：2024-04-22

Applicant: GOOGLE LLC

Inventor： Joseph Lange ,  Abhanshu Sharma ,  Adam Coimbra ,  Gökhan Bakir ,  Gabriel Taubman ,  IIya Firman ,  Jindong Chen ,  James Stout ,  Marcin Nowak-Przygodzki ,  Reed Enger ,  Thomas Weedon Hume ,  Vishwath Mohan ,  Jacek Szmigiel ,  Yunfan Jin ,  Kyle Pedersen ,  Gilles Baechler

IPC: G10L15/22 ,  G06F3/16 ,  G06F40/247 ,  G06F40/30 ,  G10L15/18

CPC classification number: G10L15/22 ,  G06F3/167 ,  G06F40/247 ,  G06F40/30 ,  G10L15/1815 ,  G10L15/1822 ,  G10L2015/223 ,  G10L2015/228

Abstract: Implementations set forth herein relate to an automated assistant that can interact with applications that may not have been pre-configured for interfacing with the automated assistant. The automated assistant can identify content of an application interface of the application to determine synonymous terms that a user may speak when commanding the automated assistant to perform certain tasks. Speech processing operations employed by the automated assistant can be biased towards these synonymous terms when the user is accessing an application interface of the application. In some implementations, the synonymous terms can be identified in a responsive language of the automated assistant when the content of the application interface is being rendered in a different language. This can allow the automated assistant to operate as an interface between the user and certain applications that may not be rendering content in a native language of the user.

公开(公告)号：US20240152369A1

公开(公告)日：2024-05-09

申请号：US18417681

申请日：2024-01-19

Applicant: GOOGLE LLC

Inventor： Prithvi Balaram ,  Nikhil Rao ,  Adam Coimbra ,  Ian Baker

IPC: G06F9/451 ,  G06F3/16 ,  G06F11/34 ,  G06F16/245

CPC classification number: G06F9/453 ,  G06F3/167 ,  G06F11/3438 ,  G06F16/245

Abstract: Interface and mode selection for digital action execution is provided. For example, a system loads a script library embedded in an electronic resource. The system determines a historic level of engagement between a client computing device and one or more digital assistants. The system selects, based on a first property of the client computing device and the historic level of engagement, a type of digital interface. The system generates, based on the type of digital interface, a digital interface with the call-to-action. The system determines, responsive to an instruction to execute the call-to-action, a mode of execution. The system selects a digital assistant and a second client device to execute the call-to-action. The system transmits the call-to-action to the second client device for execution.

公开(公告)号：US11922193B2

公开(公告)日：2024-03-05

申请号：US16643648

申请日：2020-02-28

Applicant: GOOGLE LLC

Inventor： Prithvi Balaram ,  Nikhil Rao ,  Ian Baker ,  Adam Coimbra

IPC: G06F9/451 ,  G06F3/16 ,  G06F11/34 ,  G06F16/245

CPC classification number: G06F9/453 ,  G06F3/167 ,  G06F11/3438 ,  G06F16/245

Abstract: Interface and mode selection for digital action execution is provided. For example, a system loads a script library embedded in an electronic resource. The system determines a historic level of engagement between a client computing device and one or more digital assistants. The system selects, based on a first property of the client computing device and the historic level of engagement, a type of digital interface. The system generates, based on the type of digital interface, a digital interface with the call-to-action. The system determines, responsive to an instruction to execute the call-to-action, a mode of execution. The system selects a digital assistant and a second client device to execute the call-to-action. The system transmits the call-to-action to the second client device for execution.

公开(公告)号：US11664025B2

公开(公告)日：2023-05-30

申请号：US17334416

申请日：2021-05-28

Applicant: GOOGLE LLC

Inventor： Gaurav Bhaya ,  Ulas Kirazci ,  Bradley Abrams ,  Adam Coimbra ,  Ilya Firman ,  Carey Radebaugh

IPC: G10L15/22 ,  G10L15/18 ,  G10L15/30 ,  G06F40/216 ,  G06F40/30 ,  G06F16/901 ,  G06F40/35

CPC classification number: G10L15/22 ,  G06F40/216 ,  G10L15/1815 ,  G10L15/1822 ,  G10L15/30 ,  G06F16/9024 ,  G06F16/9027 ,  G06F40/30 ,  G06F40/35 ,  G10L2015/223 ,  G10L2015/225

Abstract: The present disclosure is generally directed to the generation of voice-activated data flows in interconnected network. The voice-activated data flows can include input audio signals that include a request and are detected at a client device. The client device can transmit the input audio signal to a data processing system, where the input audio signal can be parsed and passed to the data processing system of a service provider to fulfill the request in the input audio signal. The present solution is configured to conserve network resources by reducing the number of network transmissions needed to fulfill a request.

公开(公告)号：US11574634B2

公开(公告)日：2023-02-07

申请号：US16722694

申请日：2019-12-20

Applicant: Google LLC

Inventor： Quazi Hussain ,  Adam Coimbra ,  Ilya Firman

IPC: G10L15/22 ,  G06F9/451 ,  G06F9/54 ,  G10L15/18 ,  G06F3/16 ,  G06N20/00

Abstract: Dynamic interfacing with applications is provided. For example, a system receives a first input audio signal. The system processes, via a natural language processing technique, the first input audio signal to identify an application. The system activates the application for execution on the client computing device. The application declares a function the application is configured to perform. The system modifies the natural language processing technique responsive to the function declared by the application. The system receives a second input audio signal. The system processes, via the modified natural language processing technique, the second input audio signal to detect one or more parameters. The system determines that the one or more parameters are compatible for input into an input field of the application. The system generates an action data structure for the application. The system inputs the action data structure into the application, which executes the action data structure.

公开(公告)号：US20210358489A1

公开(公告)日：2021-11-18

申请号：US16622805

申请日：2019-11-27

Applicant: Google LLC

Inventor： Quazi Hussain ,  Adam Coimbra ,  Ilya Firman

IPC: G10L15/18 ,  G10L15/06 ,  G10L15/08

Abstract: Dynamic interfacing with applications is provided. For example, a system receives a first input audio signal. The system processes, via a natural language processing technique, the first input audio signal to identify an application. The system activates the application for execution on the client computing device. The application declares a function the application is configured to perform. The system modifies the natural language processing technique responsive to the function declared by the application. The system receives a second input audio signal. The system processes, via the modified natural language processing technique, the second input audio signal to detect one or more parameters. The system determines that the one or more parameters are compatible for input into an input field of the application. The system generates an action data structure for the application. The system inputs the action data structure into the application, which executes the action data structure.

公开(公告)号：US11170772B2

公开(公告)日：2021-11-09

申请号：US16269275

申请日：2019-02-06

Applicant: Google LLC

Inventor： Ulas Kirazci ,  Adam Coimbra ,  Abraham Lee ,  Wei Dong ,  Thushan Amarasiriwardena ,  Yudong Sun ,  Xiao Gao

IPC: G10L15/22 ,  G06F3/0485 ,  G06F3/16 ,  G10L13/02

Abstract: Techniques are described herein for multi-modal interaction between users, automated assistants, and other computing services. In various implementations, a user may engage with the automated assistant in order to further engage with a third party computing service. In some implementations, the user may advance through dialog state machines associated with third party computing service using both verbal input modalities and input modalities other than verbal modalities, such as visual/tactile modalities.