公开(公告)号：US11514896B2

公开(公告)日：2022-11-29

申请号：US16622805

申请日：2019-11-27

Applicant: Google LLC

Inventor： Quazi Hussain ,  Adam Coimbra ,  Ilya Firman

IPC: G10L15/18 ,  G10L15/06 ,  G10L15/08

Abstract: Dynamic interfacing with applications is provided. For example, a system receives a first input audio signal. The system processes, via a natural language processing technique, the first input audio signal to identify an application. The system activates the application for execution on the client computing device. The application declares a function the application is configured to perform. The system modifies the natural language processing technique responsive to the function declared by the application. The system receives a second input audio signal. The system processes, via the modified natural language processing technique, the second input audio signal to detect one or more parameters. The system determines that the one or more parameters are compatible for input into an input field of the application. The system generates an action data structure for the application. The system inputs the action data structure into the application, which executes the action data structure.

公开(公告)号：US11200893B2

公开(公告)日：2021-12-14

申请号：US16269275

申请日：2019-02-06

Applicant: Google LLC

Inventor： Ulas Kirazci ,  Adam Coimbra ,  Abraham Lee ,  Wei Dong ,  Thushan Amarasiriwardena ,  Yudong Sun ,  Xiao Gao

IPC: G10L15/22 ,  G06F3/0485 ,  G06F3/16 ,  G10L13/02

Abstract: Techniques are described herein for multi-modal interaction between users, automated assistants, and other computing services. In various implementations, a user may engage with the automated assistant in order to further engage with a third party computing service. In some implementations, the user may advance through dialog state machines associated with third party computing service using both verbal input modalities and input modalities other than verbal modalities, such as visual/tactile modalities.

公开(公告)号：US11024306B2

公开(公告)日：2021-06-01

申请号：US16131453

申请日：2018-09-14

Applicant: Google LLC

Inventor： Gaurav Bhaya ,  Ulas Kirazci ,  Bradley Abrams ,  Adam Coimbra ,  Ilya Firman ,  Carey Radebaugh

IPC: G10L21/00 ,  G10L15/22 ,  G10L15/18 ,  G10L15/30 ,  G06F40/30 ,  G06F16/901 ,  G06F40/35

Abstract: The present disclosure is generally directed to the generation of voice-activated data flows in interconnected network. The voice-activated data flows can include input audio signals that include a request and are detected at a client device. The client device can transmit the input audio signal to a data processing system, where the input audio signal can be parsed and passed to the data processing system of a service provider to fulfill the request in the input audio signal. The present solution is configured to conserve network resources by reducing the number of network transmissions needed to fulfill a request.

公开(公告)号：US10984786B2

公开(公告)日：2021-04-20

申请号：US15774950

申请日：2018-05-07

Applicant: Google LLC

Inventor： Ulas Kirazci ,  Adam Coimbra ,  Abraham Lee ,  Wei Dong ,  Thushan Amarasiriwardena

IPC: G10L15/22 ,  G06F9/448 ,  G06F3/16 ,  G10L13/027

Abstract: Techniques are described herein for multi-modal interaction between users, automated assistants, and other computing services. In various implementations, a user may engage with the automated assistant in order to further engage with a third party computing service. In some implementations, the user may advance through dialog state machines associated with third party computing service using both verbal input modalities and input modalities other than verbal modalities, such as visual/tactile modalities.

公开(公告)号：US20190340200A1

公开(公告)日：2019-11-07

申请号：US16240609

申请日：2019-01-04

Applicant: Google LLC

Inventor： Adam Coimbra ,  Ulas Kirazci ,  Abraham Lee ,  Wei Dong ,  Thushan Amarasiriwardena

IPC: G06F16/9032 ,  G10L15/22 ,  G10L13/02 ,  G06F3/0485

Abstract: Techniques are described herein for multi-modal interaction between users, automated assistants, and other computing services. In various implementations, a user may engage with the automated assistant in order to further engage with a third party computing service. In some implementations, the user may advance through dialog state machines associated with third party computing service using both verbal input modalities and input modalities other than verbal modalities, such as visual/tactile modalities.

公开(公告)号：US12271742B2

公开(公告)日：2025-04-08

申请号：US18230561

申请日：2023-08-04

Applicant: GOOGLE LLC

Inventor： Cliff Kuang ,  Diana Avram ,  Mugurel-Ionut Andreica ,  Radu Voroneanu ,  Sneha Ashok ,  Deepak Goyal ,  Kyunghoon Lee ,  Alice Liang ,  Dana Ritter ,  Adam Coimbra ,  Anton Berezin ,  Andre Elisseeff

IPC: G06F9/451 ,  G06F3/0482 ,  G06F3/0484

Abstract: Implementations relate to determining a rendering type for an application that is executing automatically. Based on user interactions with an application that is associated with specified input from the user while the user is interacting with the application, a confidence metric is generated for each specified input and a rendering type is determined based on the confidence metrics. Subsequently, when the user requests that a sequence of actions be performed, the application will be displayed according to the rendering type.

公开(公告)号：US20250103648A1

公开(公告)日：2025-03-27

申请号：US18373192

申请日：2023-09-26

Applicant: Google LLC

Inventor： Keun Soo Yim ,  Adam Coimbra

IPC: G06F16/632 ,  G06F16/683 ,  G10L15/22

Abstract: Some implementations receive audio data capturing a verb-based succinct query that includes an action but is void of any app entity and determine, based on processing of the audio data, the action and one or more instance candidates associated with the action. Some of those implementations further query an app entity database for usage information and/or capability information of one or more app entities that correspond to the one or more instance candidates and generate, based on selecting from the one or more app entities, a response or action responsive to the verb-based succinct query.

公开(公告)号：US20250045079A1

公开(公告)日：2025-02-06

申请号：US18230566

申请日：2023-08-04

Applicant: GOOGLE LLC

Inventor： Diana Avram ,  Mugurel-Ionut Andreica ,  Andrea D'olimpio ,  Bogdan Prisacari ,  Felix Weissenberger ,  Andre Elisseeff ,  Cliff Kuang ,  Dana Ritter ,  Adam Coimbra

IPC: G06F9/451

Abstract: Implementations relate to identifying actions performed by a user while the user is interacting with an application and providing a routine suggestion to the user based on the identified actions. While a user is interacting with an application, screenshots of the user actions are captured and processed to determine what actions were performed by the user. The identified actions are compared to one or more template routines and a template routine is selected that matches the actions and intent of the user and provided to the user as a suggested routine. The suggested routine can be implemented by an automated assistant to perform the actions of the template by providing a corresponding command.

公开(公告)号：US20250045071A1

公开(公告)日：2025-02-06

申请号：US18230561

申请日：2023-08-04

Applicant: GOOGLE LLC

Inventor： Cliff Kuang ,  Diana Avram ,  Mugurel-Ionut Andreica ,  Radu Voroneanu ,  Sneha Ashok ,  Deepak Goyal ,  Kyunghoon Lee ,  Alice Liang ,  Dana Ritter ,  Adam Coimbra ,  Anton Berezin ,  Andre Elisseeff

IPC: G06F9/451 ,  G06F3/0482 ,  G06F3/0484

Abstract: Implementations relate to determining a rendering type for an application that is executing automatically. Based on user interactions with an application that is associated with specified input from the user while the user is interacting with the application, a confidence metric is generated for each specified input and a rendering type is determined based on the confidence metrics. Subsequently, when the user requests that a sequence of actions be performed, the application will be displayed according to the rendering type.

公开(公告)号：US12125486B2

公开(公告)日：2024-10-22

申请号：US18217326

申请日：2023-06-30

Applicant: GOOGLE LLC

Inventor： Ulas Kirazci ,  Adam Coimbra ,  Abraham Lee ,  Wei Dong ,  Thushan Amarasiriwardena

IPC: G10L15/22 ,  G06F3/16 ,  G06F9/448 ,  G10L13/027

CPC classification number: G10L15/22 ,  G06F3/167 ,  G06F9/4498 ,  G10L13/027 ,  G10L2015/223 ,  G10L2015/228

Abstract: Techniques are described herein for multi-modal interaction between users, automated assistants, and other computing services. In various implementations, a user may engage with the automated assistant in order to further engage with a third party computing service. In some implementations, the user may advance through dialog state machines associated with third party computing service using both verbal input modalities and input modalities other than verbal modalities, such as visual/tactile modalities.