公开(公告)号：US11682397B2

公开(公告)日：2023-06-20

申请号：US17484184

申请日：2021-09-24

Applicant: Google LLC

Inventor： Yuzhao Ni

IPC: H04L51/21 ,  H04L51/222 ,  H04L51/224 ,  G10L15/00 ,  G10L15/16 ,  G10L15/22 ,  G06N3/006 ,  G10L15/08 ,  G10L15/30 ,  H04L51/02 ,  H04L67/12

CPC classification number: G10L15/22 ,  G06N3/006 ,  G10L15/08 ,  G10L15/30 ,  H04L51/02 ,  H04L67/12 ,  G10L2015/088 ,  G10L2015/223

Abstract: Transferring (e.g., automatically) an automated assistant routine between client devices during execution of the automated assistant routine. The automated assistant routine can correspond to a set of actions to be performed by one or more agents and/or one or more devices. While content, corresponding to an action of the routine, is being rendered at a particular device, the user may walk away from the particular device and toward a separate device. The automated assistant routine can be automatically transferred in response, and the separate device can continue to rendering the content for the user.

公开(公告)号：US11675844B2

公开(公告)日：2023-06-13

申请号：US17582847

申请日：2022-01-24

Applicant: Google LLC

Inventor： Yuzhao Ni

IPC: G06F16/9032 ,  G10L15/26 ,  G06N3/02 ,  G10L15/22 ,  G06F16/33 ,  G06N3/006 ,  G06N3/084 ,  G06F40/35 ,  G06F40/216 ,  G10L15/18 ,  G06N5/022

CPC classification number: G06F16/90332 ,  G06F16/3334 ,  G06F40/216 ,  G06F40/35 ,  G06N3/006 ,  G06N3/02 ,  G06N3/084 ,  G10L15/18 ,  G10L15/22 ,  G10L15/26 ,  G06N5/022 ,  G10L2015/223 ,  G10L2015/227 ,  G10L2015/228

Abstract: Generating and/or recommending command bundles for a user of an automated assistant. A command bundle comprises a plurality of discrete actions that can be performed by an automated assistant. One or more of the actions of a command bundle can cause transmission of a corresponding command and/or other data to one or more devices and/or agents that are distinct from devices and/or agents to which data is transmitted based on other action(s) of the bundle. Implementations determine command bundles that are likely relevant to a user, and present those command bundles as suggestions to the user. In some of those implementations, a machine learning model is utilized to generate a user action embedding for the user, and a command bundle embedding for each of a plurality of command bundles. Command bundle(s) can be selected for suggestion based on comparison of the user action embedding and the command bundle embeddings.

公开(公告)号：US20230056213A1

公开(公告)日：2023-02-23

申请号：US17979556

申请日：2022-11-02

Applicant: GOOGLE LLC

Inventor： Yuzhao Ni ,  David Schairer

IPC: G06F9/48 ,  G10L15/22 ,  G06Q10/02 ,  G06F3/16 ,  G06F40/205 ,  G10L15/26

Abstract: Methods, apparatus, systems, and computer-readable media for engaging an automated assistant to perform multiple tasks through a multitask command. The multitask command can be a command that, when provided by a user, causes the automated assistant to invoke multiple different agent modules for performing tasks to complete the multitask command. During execution of the multitask command, a user can provide input that can be used by one or more agent modules to perform their respective tasks. Furthermore, feedback from one or more agent modules can be used by the automated assistant to dynamically alter tasks in order to more effectively use resources available during completion of the multitask command.

公开(公告)号：US20220405122A1

公开(公告)日：2022-12-22

申请号：US17892672

申请日：2022-08-22

Applicant: GOOGLE LLC

Inventor： Yuzhao Ni

IPC: G06F9/48 ,  G06F9/451 ,  G06N5/04 ,  G10L15/18 ,  G10L15/22 ,  G10L15/30 ,  H04L51/02

Abstract: Methods, apparatus, systems, and computer-readable media are provided for allowing an automated assistant routine to be interrupted during performance of the routine. A routine can correspond to a set of actions to be performed at the direction of the automated assistant. When the routine is initialized and a user subsequently issues a command to interrupt the routine, the automated assistant can modify a status identifier for the routine. That status identifier can be stored at a database and allow other applications and/or devices that are operating to complete the routine to be put on notice that the user has requested the routine be interrupted. The database can be accessible to one or more devices and/or applications, such as third party applications, in order to provide a medium through which the devices and/or applications can check the statuses of various automated assistant routines.

公开(公告)号：US20220197593A1

公开(公告)日：2022-06-23

申请号：US17691889

申请日：2022-03-10

Applicant: GOOGLE LLC

Inventor： Tomer Amarilio ,  Yuzhao Ni ,  Bryan Allen ,  Norbert Tydingco ,  Will Donnelly ,  Feng Yuan ,  Nathaniel Nesiba ,  Anurag Jain ,  Jacky Cheung ,  Ronghui Zhu ,  Chunya Hua ,  Gregory Kielian

IPC: G06F3/16 ,  G10L15/08 ,  G10L15/22

Abstract: In response to a user interacting with a tangible peripheral assistant control device (e.g., depressing a button of the device), causing an automated assistant to perform one or more actions. The action(s) performed can be based on input previously provided by the user in configuring the peripheral assistant control device. The action(s) performed in response to interaction with the peripheral assistant control device can vary based on one or more conditions, such as which user is currently active, where the peripheral assistant control device is currently located (which can optionally be inferred based on which of multiple assistant computing devices the button is paired with), and/or the current state of one or more smart devices and/or other devices (e.g., as determined based on a device topology). A utility of the peripheral assistant control device can be automatically extended beyond what was specifically requested by a user during configuration.

公开(公告)号：US20220094564A1

公开(公告)日：2022-03-24

申请号：US17543069

申请日：2021-12-06

Applicant: Google LLC

Inventor： Yuzhao Ni

IPC: H04L12/28 ,  G10L15/22 ,  G10L15/30

Abstract: Implementations herein relate to information describing one or more internal states of a technical system. Implementations herein are provided for characterizing reliability of various different third party servers, at least when reporting third party device statuses, as well as adapting protocols for device ecosystems affected by such reliability. Latency can affect accuracy of device states represented by assistant devices. Certain servers can be characterized as especially delayed when reporting an updated device state in response to a user request, and, as a result, the third party server can be correlated to a metric that characterizes the relative latency of the third party server. When the metric fails to satisfy a particular threshold, a server and/or client associated with the “ecosystem” of third party devices can affirmatively operate to retrieve device state updates, rather than passively await updates from a corresponding third party server.

公开(公告)号：US11269305B2

公开(公告)日：2022-03-08

申请号：US16617994

申请日：2019-03-13

Applicant: Google LLC

Inventor： Yuzhao Ni ,  David Roy Schairer

IPC: H04L29/08 ,  G05B19/042 ,  G06F9/451 ,  G06F3/04847 ,  H04L12/28 ,  H04L67/53

Abstract: Efficient control and/or linking of smart network connected devices. Some implementations are directed to rendering, within an active application that enables control of multiple smart devices from multiple parties, both: (1) an interactive graphical interface element that, when interacted with, causes corresponding control of smart device(s) of a third-party (3P); and (2) a selectable element that, when selected, causes execution of a deeplink to cause a 3P application to open in a state that presents additional information for the smart device(s). Some implementations are additionally or alternatively directed to, in response to determining that a application is installed at a client device, but a smart device topology lacks any entries for 3P smart devices of the 3P, efficiently adding one or more particular smart devices of the 3P to the smart device topology.

公开(公告)号：US20210281437A1

公开(公告)日：2021-09-09

申请号：US16343927

申请日：2019-02-08

Applicant: Google LLC

Inventor： Yuzhao Ni

IPC: H04L12/28 ,  G10L15/30 ,  G10L15/22

Abstract: Implementations herein relate to information describing one or more internal states of a technical system. Implementations herein are provided for characterizing reliability of various different third party servers, at least when reporting third party device statuses, as well as adapting protocols for device ecosystems affected by such reliability. Latency can affect accuracy of device states represented by assistant devices. Certain servers can be characterized as especially delayed when reporting an updated device state in response to a user request, and, as a result, the third party server can be correlated to a metric that characterizes the relative latency of the third party server. When the metric fails to satisfy a particular threshold, a server and/or client associated with the “ecosystem” of third party devices can affirmatively operate to retrieve device state updates, rather than passively await updates from a corresponding third party server.

公开(公告)号：US10985936B2

公开(公告)日：2021-04-20

申请号：US16154512

申请日：2018-10-08

Applicant: Google LLC

Inventor： Benjamin Brown ,  Christopher Conover ,  Lishan Zhang ,  Alexander Crettenand ,  Lantian Zheng ,  Mafang Yao ,  Minoo Erfani Joorabchi ,  Francisco Pedro Lopes Pimenta ,  Rui Zhao ,  Yuzhao Ni

IPC: H04L12/28 ,  G06F3/16 ,  G10L17/22

Abstract: Various arrangements for using captured voice to generate a custom interface controller are presented. A vocal recording from a user may be captured in which a spoken command and multiple smart-home devices are indicated. One or more common functions that map to the multiple smart-home devices may be determined. A custom interface controller may be generated that controls the one or more common functions of each smart-home device of the multiple smart-home devices.

公开(公告)号：US20210082436A1

公开(公告)日：2021-03-18

申请号：US17109347

申请日：2020-12-02

Applicant: Google LLC

Inventor： Yuzhao Ni ,  David Roy Schairer

IPC: G10L15/22 ,  G10L15/08 ,  G10L17/22 ,  G06F21/32 ,  H04L29/06

Abstract: In one example, a method includes receiving audio data generated by one or more microphones of a computing device, the audio data representing a spoken utterance; identifying, based on the audio data, a user that provided the spoken utterance; identifying, based on the audio data, an automation action associated with one or more automation devices, the automation action corresponding to the spoken utterance; determining whether the identified user is authorized to cause performance of the identified automation action; and responsive to determining that the identified user is authorized to cause performance of the identified automation action, causing the one or more automation devices to perform the identified automation action.