公开(公告)号：US11943075B2

公开(公告)日：2024-03-26

申请号：US17543069

申请日：2021-12-06

Applicant: Google LLC

Inventor： Yuzhao Ni

IPC: H04L12/28 ,  G10L15/22 ,  G10L15/30

CPC classification number: H04L12/283 ,  G10L15/22 ,  G10L15/30 ,  H04L12/282 ,  G10L2015/223

Abstract: Implementations herein relate to information describing one or more internal states of a technical system. Implementations herein are provided for characterizing reliability of various different third party servers, at least when reporting third party device statuses, as well as adapting protocols for device ecosystems affected by such reliability. Latency can affect accuracy of device states represented by assistant devices. Certain servers can be characterized as especially delayed when reporting an updated device state in response to a user request, and, as a result, the third party server can be correlated to a metric that characterizes the relative latency of the third party server. When the metric fails to satisfy a particular threshold, a server and/or client associated with the “ecosystem” of third party devices can affirmatively operate to retrieve device state updates, rather than passively await updates from a corresponding third party server.

公开(公告)号：US20240029735A1

公开(公告)日：2024-01-25

申请号：US18375861

申请日：2023-10-02

Applicant: Google LLC

Inventor： Yuzhao Ni ,  David Roy Schairer

IPC: G10L15/22 ,  G10L15/08 ,  G10L17/22 ,  G06F21/32 ,  H04L9/40

CPC classification number: G10L15/22 ,  G10L15/08 ,  G10L17/22 ,  G06F21/32 ,  H04L63/0861 ,  H04L63/102 ,  G10L2015/223 ,  G10L2015/088

Abstract: In one example, a method includes receiving audio data generated by one or more microphones of a computing device, the audio data representing a spoken utterance; identifying, based on the audio data, a user that provided the spoken utterance; identifying, based on the audio data, an automation action associated with one or more automation devices, the automation action corresponding to the spoken utterance; determining whether the identified user is authorized to cause performance of the identified automation action; and responsive to determining that the identified user is authorized to cause performance of the identified automation action, causing the one or more automation devices to perform the identified automation action.

公开(公告)号：US11861393B2

公开(公告)日：2024-01-02

申请号：US17979556

申请日：2022-11-02

Applicant: GOOGLE LLC

Inventor： Yuzhao Ni ,  David Schairer

IPC: G10L15/19 ,  G06F9/48 ,  G10L15/22 ,  G06Q10/02 ,  G06F3/16 ,  G06F40/205 ,  G10L15/26

CPC classification number: G06F9/4831 ,  G06F3/16 ,  G06F3/167 ,  G06F9/4843 ,  G06F40/205 ,  G06Q10/02 ,  G10L15/22 ,  G10L15/26 ,  G10L2015/223

Abstract: Methods, apparatus, systems, and computer-readable media for engaging an automated assistant to perform multiple tasks through a multitask command. The multitask command can be a command that, when provided by a user, causes the automated assistant to invoke multiple different agent modules for performing tasks to complete the multitask command. During execution of the multitask command, a user can provide input that can be used by one or more agent modules to perform their respective tasks. Furthermore, feedback from one or more agent modules can be used by the automated assistant to dynamically alter tasks in order to more effectively use resources available during completion of the multitask command.

公开(公告)号：US11720635B2

公开(公告)日：2023-08-08

申请号：US17582847

申请日：2022-01-24

Applicant: Google LLC

Inventor： Yuzhao Ni

IPC: G06F16/9032 ,  G10L15/26 ,  G06N3/02 ,  G10L15/22 ,  G06F16/33 ,  G06N3/006 ,  G06N3/084 ,  G06F40/35 ,  G06F40/216 ,  G10L15/18 ,  G06N5/022

CPC classification number: G06F16/90332 ,  G06F16/3334 ,  G06F40/216 ,  G06F40/35 ,  G06N3/006 ,  G06N3/02 ,  G06N3/084 ,  G10L15/18 ,  G10L15/22 ,  G10L15/26 ,  G06N5/022 ,  G10L2015/223 ,  G10L2015/227 ,  G10L2015/228

Abstract: Generating and/or recommending command bundles for a user of an automated assistant. A command bundle comprises a plurality of discrete actions that can be performed by an automated assistant. One or more of the actions of a command bundle can cause transmission of a corresponding command and/or other data to one or more devices and/or agents that are distinct from devices and/or agents to which data is transmitted based on other action(s) of the bundle. Implementations determine command bundles that are likely relevant to a user, and present those command bundles as suggestions to the user. In some of those implementations, a machine learning model is utilized to generate a user action embedding for the user, and a command bundle embedding for each of a plurality of command bundles. Command bundle(s) can be selected for suggestion based on comparison of the user action embedding and the command bundle embeddings.

公开(公告)号：US11579749B2

公开(公告)日：2023-02-14

申请号：US16532962

申请日：2019-08-06

Applicant: Google LLC

Inventor： Yuzhao Ni ,  David Roy Schairer

IPC: G06F3/0482 ,  G06F3/0484 ,  G10L15/22 ,  H04L12/28 ,  H04L67/12 ,  G06F3/04812

Abstract: Methods, apparatus, systems, and computer-readable media are provided for tailoring composite graphical assistant interfaces for interacting with multiple different connected devices. The composite graphical assistant interfaces can be generated proactively and/or in response to a user providing a request for an automated assistant to cause a connected device to perform a particular function. In response to the automated assistant receiving the request, the automated assistant can identify other connected devices, and other functions capable of being performed by the other connected devices. The other functions can then be mapped to various graphical control elements in order to provide a composite graphical assistant interface from which the user can interact with different connected devices. Each graphical control element can be arranged to reflect how each connected device is operating simultaneous to the presentation of the composite graphical assistant interface.

公开(公告)号：US20230013457A1

公开(公告)日：2023-01-19

申请号：US17946909

申请日：2022-09-16

Applicant: GOOGLE LLC

Inventor： Yuzhao Ni ,  Lucas Palmer

IPC: G10L15/22 ,  G10L15/30 ,  G10L15/18

Abstract: Methods, apparatus, systems, and computer-readable media are provided for using shortcut command phrases to operate an automated assistant. A user of the automated assistant can request that a shortcut command phrase be established for causing the automated assistant to perform a variety of different actions. In this way, the user does not necessarily have to provide an individual command for each action to be performed but, rather, can use a shortcut command phrase to cause the automated assistant to perform the actions. The shortcut command phrases can be used to control peripheral devices, IoT devices, applications, websites, and/or any other apparatuses or processes capable of being controlled through an automated assistant.

公开(公告)号：US11422848B2

公开(公告)日：2022-08-23

申请号：US16337739

申请日：2018-01-12

Applicant: Google LLC

Inventor： Yuzhao Ni

IPC: G06F9/46 ,  G06F9/48 ,  G06F9/451 ,  G06N5/04 ,  G10L15/18 ,  G10L15/22 ,  G10L15/30 ,  H04L51/02 ,  G06N20/00 ,  G16Y10/75

Abstract: Methods, apparatus, systems, and computer-readable media are provided for allowing an automated assistant routine to be interrupted during performance of the routine. A routine can correspond to a set of actions to be performed at the direction of the automated assistant. When the routine is initialized and a user subsequently issues a command to interrupt the routine, the automated assistant can modify a status identifier for the routine. That status identifier can be stored at a database and allow other applications and/or devices that are operating to complete the routine to be put on notice that the user has requested the routine be interrupted. The database can be accessible to one or more devices and/or applications, such as third party applications, in order to provide a medium through which the devices and/or applications can check the statuses of various automated assistant routines.

公开(公告)号：US11256405B2

公开(公告)日：2022-02-22

申请号：US16339269

申请日：2018-09-24

Applicant: Google LLC

Inventor： Yuzhao Ni

IPC: G06F3/0484 ,  G06F3/04847 ,  H04N5/232 ,  G06F3/04883 ,  G06V20/10

Abstract: Implementations discussed herein relate to systems, methods, and apparatuses for controlling one or more internet of things (IoT) devices via a graphical camera interface of a portable electronic device. The graphical camera interface can be presented at the portable electronic device in response to the user accessing a camera application via the portable electronic device and directing a camera of the portable electronic device at the IoT device. One or more images generated from an output of the camera can be processed to identify the IoT device that the user is intending to control. Information related to a context in which the user aimed their camera at the IoT device can be used to assist in narrowing down a selection of the IoT device from multiple different IoT devices that may be associated with the user and/or an image captured by the camera.

公开(公告)号：US11237796B2

公开(公告)日：2022-02-01

申请号：US16621080

申请日：2018-05-07

Applicant: Google LLC

Inventor： David Roy Schairer ,  Triona Butler ,  Cindy Tran ,  Mark Spates, IV ,  Di Lin ,  Yuzhao Ni ,  Lisa Williams

IPC: G06F3/16 ,  G06F3/0484 ,  G06F3/0488 ,  G10L15/22 ,  H04L12/58

Abstract: Methods, apparatus, systems, and computer-readable media are provided for tailoring composite graphical assistant interfaces for interacting with multiple different connected devices. The composite graphical assistant interfaces can be generated in response to a user providing a request for an automated assistant to cause a connected device to perform a particular function. In response to the automated assistant receiving the request, the automated assistant can identify other functions that the connected device is capable of performing. The other functions can then be mapped to various graphical control elements in order to provide a composite graphical assistant interface from which the user can interact with the connected device. Each graphical control element can be arranged according to a status of the connected device, in order to reflect how the connected device is operating simultaneous to the presentation of the composite graphical assistant interface.

公开(公告)号：US11177044B2

公开(公告)日：2021-11-16

申请号：US16971542

申请日：2019-10-08

Applicant: Google LLC

Inventor： Yuzhao Ni ,  Ashwin Limaye ,  Cindy Tran ,  Thomas Clifton ,  David Roy Schairer

IPC: G16Y40/35 ,  G16Y40/10 ,  G16Y10/80 ,  G16Y20/20 ,  G16Y20/40 ,  G10L15/22

Abstract: Implementing methods to provide a shortened textual summary that includes status information that is most pertinent to the user for a plurality of connected smart appliances. The method includes determining a list of current statuses for a plurality of enabled smart appliances and filtering the list to remove statuses that may not be of interest to the user. The filtering of the list is based on a current context of the requesting user and one or more previous contexts of the user. The resulting filtered statuses are then converted to textual snippets, summarized, and provided to the user via one or more output devices.