公开(公告)号：US20240233373A1

公开(公告)日：2024-07-11

申请号：US18444736

申请日：2024-02-18

Applicant: GOOGLE LLC

Inventor： Marcin Nowak-Przygodzki ,  Gökhan Bakir

IPC: G06V20/20 ,  G06F3/00 ,  G06F3/01 ,  G06F3/03 ,  G06F3/0481 ,  G06F3/16 ,  G06F9/451 ,  G06F16/58 ,  G06F16/9032 ,  H04N23/63

CPC classification number: G06V20/20 ,  G06F3/005 ,  G06F3/017 ,  G06F3/0304 ,  G06F3/0481 ,  G06F3/167 ,  G06F9/453 ,  G06F16/5866 ,  G06F16/9032 ,  H04N23/63

Abstract: Generating and/or utilizing image shortcuts that cause one or more corresponding computer actions to be performed in response to determining that one or more features are present in image(s) from a camera of a computing device of a user (e.g., present in a real-time image feed from the camera). An image shortcut can be generated in response to user interface input, such as a spoken command. For example, the user interface input can direct the automated assistant to perform one or more actions in response to object(s) having certain feature(s) being present in a field of view of the camera. Subsequently, when the user directs their camera at object(s) having such feature(s), the assistant application can cause the action(s) to be automatically performed. For example, the assistant application can cause data to be presented and/or can control a remote device in accordance with the image shortcut.

公开(公告)号：US20240185857A1

公开(公告)日：2024-06-06

申请号：US18439411

申请日：2024-02-12

Applicant: GOOGLE LLC

Inventor： Denis Burakov ,  Behshad Behzadi ,  Mario Bertschlewr ,  Bohdan Vlasyuk ,  Daniel Cotting ,  Michael Golikov ,  Lucas Mirelmann ,  Steve Cheng ,  Sergey Nazarov ,  Zaheed Sabur ,  Marcin Nowak-Przygodzki ,  Mugurel Ionut Andreica ,  Radu Voroneanu

IPC: G10L15/26 ,  G06F3/16 ,  G10L15/22

CPC classification number: G10L15/26 ,  G06F3/167 ,  G10L15/22 ,  G10L2015/223

Abstract: Implementations set forth herein relate to a system that employs an automated assistant to further interactions between a user and another application, which can provide the automated assistant with permission to initialize relevant application actions simultaneous to the user interacting with the other application. Furthermore, the system can allow the automated assistant to initialize actions of different applications, despite being actively operating a particular application. Available actions can be gleaned by the automated assistant using various application-specific schemas, which can be compared with incoming requests from a user to the automated assistant. Additional data, such as context and historical interactions, can also be used to rank and identify a suitable application action to be initialized via the automated assistant.

公开(公告)号：US11908187B2

公开(公告)日：2024-02-20

申请号：US18117798

申请日：2023-03-06

Applicant: GOOGLE LLC

Inventor： Marcin Nowak-Przygodzki ,  Gökhan Bakir

IPC: H04N23/63 ,  G06F9/451 ,  G06F16/58 ,  G06V20/20 ,  G06F16/9032 ,  G06F3/01 ,  G06F3/03 ,  G06F3/00 ,  G06F3/0481 ,  G06F3/16

CPC classification number: G06V20/20 ,  G06F3/005 ,  G06F3/017 ,  G06F3/0304 ,  G06F3/0481 ,  G06F3/167 ,  G06F9/453 ,  G06F16/5866 ,  G06F16/9032 ,  H04N23/63

Abstract: Methods, apparatus, systems, and computer-readable media are set forth for generating and/or utilizing image shortcuts that cause one or more corresponding computer actions to be performed in response to determining that one or more features are present in image(s) from a camera of a computing device of a user (e.g., present in a real-time image feed from the camera). An image shortcut can be generated in response to user interface input, such as a spoken command. For example, the user interface input can direct the automated assistant to perform one or more actions in response to object(s) having certain feature(s) being present in a field of view of the camera. Subsequently, when the user directs their camera at object(s) having such feature(s), the assistant application can cause the action(s) to be automatically performed. For example, the assistant application can cause data to be presented and/or can control a remote device in accordance with the image shortcut.

公开(公告)号：US20230274733A1

公开(公告)日：2023-08-31

申请号：US18144694

申请日：2023-05-08

Applicant: GOOGLE LLC

Inventor： Marcin Nowak-Przygodzki ,  Nathan David Howard ,  Gabor Simko ,  Andrei Giurgiu ,  Behshad Behzadi

IPC: G10L15/18 ,  G10L15/07 ,  G10L25/51 ,  G06F16/9032 ,  G10L15/08 ,  G10L15/22

CPC classification number: G10L15/1815 ,  G10L15/07 ,  G10L25/51 ,  G06F16/90332 ,  G10L15/08 ,  G10L15/22 ,  G10L2015/227 ,  G10L2015/223 ,  G10L2015/088

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for detecting a continued conversation are disclosed. In one aspect, a method includes the actions of receiving first audio data of a first utterance. The actions further include obtaining a first transcription of the first utterance. The actions further include receiving second audio data of a second utterance. The actions further include obtaining a second transcription of the second utterance. The actions further include determining whether the second utterance includes a query directed to a query processing system based on analysis of the second transcription and the first transcription or a response to the first query. The actions further include configuring the data routing component to provide the second transcription of the second utterance to the query processing system as a second query or bypass routing the second transcription.

公开(公告)号：US11676582B2

公开(公告)日：2023-06-13

申请号：US17117621

申请日：2020-12-10

Applicant: Google LLC

Inventor： Marcin Nowak-Przygodzki ,  Nathan David Howard ,  Gabor Simko ,  Andrei Giurgiu ,  Behshad Behzadi

IPC: G10L15/18 ,  G10L15/07 ,  G10L25/51 ,  G06F16/9032 ,  G10L15/08 ,  G10L15/22

CPC classification number: G10L15/1815 ,  G06F16/90332 ,  G10L15/07 ,  G10L15/08 ,  G10L15/22 ,  G10L25/51 ,  G10L2015/088 ,  G10L2015/223 ,  G10L2015/227

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for detecting a continued conversation are disclosed. In one aspect, a method includes the actions of receiving first audio data of a first utterance. The actions further include obtaining a first transcription of the first utterance. The actions further include receiving second audio data of a second utterance. The actions further include obtaining a second transcription of the second utterance. The actions further include determining whether the second utterance includes a query directed to a query processing system based on analysis of the second transcription and the first transcription or a response to the first query. The actions further include configuring the data routing component to provide the second transcription of the second utterance to the query processing system as a second query or bypass routing the second transcription.

公开(公告)号：US20220392216A1

公开(公告)日：2022-12-08

申请号：US17888163

申请日：2022-08-15

Applicant: GOOGLE LLC

Inventor： Marcin Nowak-Przygodzki ,  Gökhan Bakir

IPC: G06V20/20 ,  G06F16/487 ,  G06F3/0482 ,  G06F3/01 ,  G06Q30/02 ,  G06F16/9032 ,  G06F3/04886 ,  H04N5/232

Abstract: Techniques described herein enable a user to interact with an automated assistant and obtain relevant output from the automated assistant without requiring arduous typed input to be provided by the user and/or without requiring the user to provide spoken input that could cause privacy concerns (e.g., if other individuals are nearby). The assistant application can operate in multiple different image conversation modes in which the assistant application is responsive to various objects in a field of view of the camera. The image conversation modes can be suggested to the user when a particular object is detected in the field of view of the camera. When the user selects an image conversation mode, the assistant application can thereafter provide output, for presentation, that is based on the selected image conversation mode and that is based on object(s) captured by image(s) of the camera.

公开(公告)号：US20220157317A1

公开(公告)日：2022-05-19

申请号：US17588481

申请日：2022-01-31

Applicant: Google LLC

Inventor： Denis Burakov ,  Behshad Behzadi ,  Mario Bertschler ,  Bohdan Vlasyuk ,  Daniel Cotting ,  Michael Golikov ,  Lucas Mirelmann ,  Steve Cheng ,  Sergey NAZAROV ,  Zaheed Sabur ,  Marcin Nowak-Przygodzki ,  Mugurel Ionut Andreica ,  Radu Voroneanu

IPC: G10L15/26 ,  G06F3/16 ,  G10L15/22

Abstract: Implementations set forth herein relate to a system that employs an automated assistant to further interactions between a user and another application, which can provide the automated assistant with permission to initialize relevant application actions simultaneous to the user interacting with the other application. Furthermore, the system can allow the automated assistant to initialize actions of different applications, despite being actively operating a particular application. Available actions can be gleaned by the automated assistant using various application-specific schemas, which can be compared with incoming requests from a user to the automated assistant. Additional data, such as context and historical interactions, can also be used to rank and identify a suitable application action to be initialized via the automated assistant.

公开(公告)号：US20210295841A1

公开(公告)日：2021-09-23

申请号：US17339114

申请日：2021-06-04

Applicant: Google LLC

Inventor： Mugurel Ionut Andreica ,  Vladimir Vuskovic ,  Joseph Lange ,  Sharon Stovezky ,  Marcin Nowak-Przygodzki

IPC: G10L15/22 ,  G06N3/08 ,  G10L15/02

Abstract: Implementations are set forth herein for creating an order of execution for actions that were requested by a user, via a spoken utterance to an automated assistant. The order of execution for the requested actions can be based on how each requested action can, or is predicted to, affect other requested actions. In some implementations, an order of execution for a series of actions can be determined based on an output of a machine learning model, such as a model that has been trained according to supervised learning. A particular order of execution can be selected to mitigate waste of processing, memory, and network resources—at least relative to other possible orders of execution. Using interaction data that characterizes past performances of automated assistants, certain orders of execution can be adapted over time, thereby allowing the automated assistant to learn from past interactions with one or more users.

公开(公告)号：US11031007B2

公开(公告)日：2021-06-08

申请号：US16343285

申请日：2019-02-07

Applicant: Google LLC

Inventor： Mugurel Ionut Andreica ,  Vladimir Vuskovic ,  Joseph Lange ,  Sharon Stovezky ,  Marcin Nowak-Przygodzki

IPC: G10L15/22 ,  G06N3/08 ,  G10L15/02

Abstract: Implementations are set forth herein for creating an order of execution for actions that were requested by a user, via a spoken utterance to an automated assistant. The order of execution for the requested actions can be based on how each requested action can, or is predicted to, affect other requested actions. In some implementations, an order of execution for a series of actions can be determined based on an output of a machine learning model, such as a model that has been trained according to supervised learning. A particular order of execution can be selected to mitigate waste of processing, memory, and network resources—at least relative to other possible orders of execution. Using interaction data that characterizes past performances of automated assistants, certain orders of execution can be adapted over time, thereby allowing the automated assistant to learn from past interactions with one or more users.

公开(公告)号：US20210097982A1

公开(公告)日：2021-04-01

申请号：US17117621

申请日：2020-12-10

Applicant: Google LLC

Inventor： Marcin Nowak-Przygodzki ,  Nathan David Howard ,  Gabor Simko ,  Andrei Giurgiu ,  Behshad Behzadi

IPC: G10L15/18 ,  G10L15/07 ,  G06F16/9032 ,  G10L25/51

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for detecting a continued conversation are disclosed. In one aspect, a method includes the actions of receiving first audio data of a first utterance. The actions further include obtaining a first transcription of the first utterance. The actions further include receiving second audio data of a second utterance. The actions further include obtaining a second transcription of the second utterance. The actions further include determining whether the second utterance includes a query directed to a query processing system based on analysis of the second transcription and the first transcription or a response to the first query. The actions further include configuring the data routing component to provide the second transcription of the second utterance to the query processing system as a second query or bypass routing the second transcription.