公开(公告)号：US20230143177A1

公开(公告)日：2023-05-11

申请号：US18092883

申请日：2023-01-03

Applicant: GOOGLE LLC

Inventor： Victor Carbune ,  Matthew Sharifi ,  Ondrej Skopek ,  Justin Lu ,  Daniel Valcarce ,  Kevin Kilgour ,  Mohamad Hassan Rom ,  Nicolo D'Ercole ,  Michael Golikov

IPC: G10L15/22 ,  G10L15/05 ,  G10L15/18 ,  G10L25/78

CPC classification number: G10L15/22 ,  G10L15/05 ,  G10L15/1815 ,  G10L25/78 ,  G10L2015/088

Abstract: Some implementations process, using warm word model(s), a stream of audio data to determine a portion of the audio data that corresponds to particular word(s) and/or phrase(s) (e.g., a warm word) associated with an assistant command, process, using an automatic speech recognition (ASR) model, a preamble portion of the audio data (e.g., that precedes the warm word) and/or a postamble portion of the audio data (e.g., that follows the warm word) to generate ASR output, and determine, based on processing the ASR output, whether a user intended the assistant command to be performed. Additional or alternative implementations can process the stream of audio data using a speaker identification (SID) model to determine whether the audio data is sufficient to identify the user that provided a spoken utterance captured in the stream of audio data, and determine if that user is authorized to cause performance of the assistant command.

公开(公告)号：US11574640B2

公开(公告)日：2023-02-07

申请号：US16946952

申请日：2020-07-13

Applicant: Google LLC

Inventor： Victor Carbune ,  Matthew Sharifi

IPC: G10L17/06 ,  G06V20/00 ,  G06F3/16 ,  H04L51/02 ,  G10L17/00

Abstract: Implementations set forth herein relate to an automated assistant that can be customized by a user to provide custom assistant responses to certain assistant queries, which may originate from other users. The user can establish certain custom assistant responses by providing an assistant response request to the automated assistant and/or responding to a request from the automated assistant to establish a particular custom assistant response. In some instances, a user can elect to establish a custom assistant response when the user determines or acknowledges that certain common queries are being submitted to the automated assistant—but the automated assistant is unable to resolve the common query. Establishing such custom assistant responses can therefore condense interactions between other users and the automated assistant. Furthermore, as such interactions are more immediately resolved, the automated assistant can avoid wasteful consumption of computational resources that may otherwise occur during prolonged assistant interactions.

公开(公告)号：US11573810B1

公开(公告)日：2023-02-07

申请号：US17827196

申请日：2022-05-27

Applicant: GOOGLE LLC

Inventor： Matthew Sharifi ,  David Petrou

IPC: G06F9/451 ,  G06F16/14 ,  G06F16/583 ,  G06F16/22 ,  G06F16/58 ,  G06F16/2457 ,  G06F16/957 ,  G06F40/134 ,  G06F40/169 ,  G06F40/295 ,  G06V30/416 ,  G06K9/62 ,  G06Q10/10 ,  G06F3/04842 ,  G06F3/04845 ,  H04L67/06 ,  H04W4/18

Abstract: Systems and methods are provided for sharing a screen from a mobile device. For example, a method includes receiving, at a second mobile device, an image of a screen captured from a first mobile device and determining whether to trigger an automated action. The method may also include displaying, responsive to not triggering the automated action, annotation data generated for the image with the image on a display of the second mobile device, the annotation data including at least one visual cue corresponding to content in the image relevant to a user of the second mobile device. The method may further include, responsive to triggering the automated action, determining that a mobile application associated with the image is installed on the second mobile device and replaying user input actions received with the image on the second mobile device starting from a reference screen associated with the mobile application.

公开(公告)号：US11562748B2

公开(公告)日：2023-01-24

申请号：US17108705

申请日：2020-12-01

Applicant: Google LLC

Inventor： Matthew Sharifi ,  Victor Carbune

IPC: G10L17/22 ,  G06F3/16 ,  G10L19/018 ,  G10L25/51

Abstract: Techniques are described herein for detecting and suppressing commands in media that may trigger another automated assistant. A method includes: determining, for each of a plurality of automated assistant devices in an environment that are each executing at least one automated assistant, an active capability of the automated assistant device; initiating playback of digital media by an automated assistant; in response to initiating playback, processing the digital media to identify an audio segment in the digital media that, upon playback, is expected to trigger activation of at least one automated assistant executing on at least one of the plurality of automated assistant devices in the environment, based on the active capability of the at least one of the plurality of automated assistant devices; and in response to identifying the audio segment in the digital media, modifying the digital media to suppress the activation of the at least one automated assistant.

公开(公告)号：US11557300B2

公开(公告)日：2023-01-17

申请号：US17087358

申请日：2020-11-02

Applicant: Google LLC

Inventor： Victor Carbune ,  Matthew Sharifi

IPC: G10L15/32 ,  G10L15/22

Abstract: Techniques are described herein for detecting and handling failures in other automated assistants. A method includes: executing a first automated assistant in an inactive state at least in part on a computing device operated by a user; while in the inactive state, determining, by the first automated assistant, that a second automated assistant failed to fulfill a request of the user; in response to determining that the second automated assistant failed to fulfill the request of the user, the first automated assistant processing cached audio data that captures a spoken utterance of the user comprising the request that the second automated assistant failed to fulfill, or features of the cached audio data, to determine a response that fulfills the request of the user; and providing, by the first automated assistant to the user, the response that fulfills the request of the user.

公开(公告)号：US11539992B2

公开(公告)日：2022-12-27

申请号：US15908481

申请日：2018-02-28

Applicant: Google LLC

Inventor： Jakob Foerster ,  Matthew Sharifi

IPC: H04N21/2387 ,  H04N21/25 ,  H04N21/845 ,  H04N21/6587 ,  H04N21/442 ,  H04N21/24 ,  H04N21/6543

Abstract: Implementations disclose methods and systems for providing a media item at an adjusted playback. A method includes receiving, from a first user device, a playback request from a first user for a first media item including one or more portions of media content; determining an adjusted playback for at least one portion of the first media item that is different than a default playback for the at least one portion of the first media item. The determining is based on previous playback behavior of one or more users in relation to one or more media items that each included one or more portions of media content corresponding to the one or more portions media content of the first media item; and causing the at least one portion of the first media item to be rendered on the first user device at the adjusted playback.

公开(公告)号：US11531453B2

公开(公告)日：2022-12-20

申请号：US15433587

申请日：2017-02-15

Applicant: Google LLC

Inventor： Matthew Sharifi ,  Jakob Nicolaus Foerster

IPC: H04L51/02 ,  G06F3/04842 ,  H04L51/04 ,  G06N20/00 ,  G06Q10/10 ,  G06F40/166 ,  H04L51/42

Abstract: A system and method for grouping and organizing structured responses in a communication application at a computing device. A structured question in a plurality of messages can be detected based on a structured question model trained via machine learning. A structured question can be a question predicted by the structured question model to have a number of possible answers fewer than a threshold. A user interface element, corresponding to the structured question, can include a structured summarization that includes one or more answers to the structured question present in the plurality of messages from the plurality of users, and/or a structured response template in which at least a subset of possible answers are presented and are selectable. A command to include the generated graphical user interface element in a record of the communication session in a graphical user interface corresponding to the communication application.

公开(公告)号：US11521597B2

公开(公告)日：2022-12-06

申请号：US17011606

申请日：2020-09-03

Applicant: Google LLC

Inventor： Matthew Sharifi ,  Victor Carbune

IPC: G10L15/00 ,  G10L15/08 ,  G06F3/16 ,  G10L13/02

Abstract: Implementations can receive audio data corresponding to a spoken utterance of a user, process the audio data to generate a plurality of speech hypotheses, determine an action to be performed by an automated assistant based on the speech hypotheses, and cause the computing device to render an indication of the action. In response to the computing device rendering the indication, implementations can receive additional audio data corresponding to an additional spoken utterance of the user, process the additional audio data to determine that a portion of the spoken utterance is similar to an additional portion of the additional spoken utterance, supplant the action with an alternate action, and cause the automated assistant to initiate performance of the alternate action. Some implementations can determine whether to render the indication of the action based on a confidence level associated with the action.

公开(公告)号：US20220382802A1

公开(公告)日：2022-12-01

申请号：US17336000

申请日：2021-06-01

Applicant: GOOGLE LLC

Inventor： Matthew Sharifi ,  Victor Carbune

IPC: G06F16/532 ,  G06T3/40

Abstract: Techniques are described herein for providing smart suggestions for image zoom regions. A method includes: receiving a search query; performing a search using the search query to identify search results that include image search results including a plurality of images that are responsive to the search query; for a given image of the plurality of images included in the image search results, determining at least one zoom region in the given image; and providing the search results including the image search results, including providing the given image and an indication of the at least one zoom region in the given image.

公开(公告)号：US20220366903A1

公开(公告)日：2022-11-17

申请号：US17321994

申请日：2021-05-17

Applicant: GOOGLE LLC

Inventor： Victor Carbune ,  Matthew Sharifi ,  Ondrej Skopek ,  Justin Lu ,  Daniel Valcarce ,  Kevin Kilgour ,  Mohamad Hassan Rom ,  Nicolo D'Ercole ,  Michael Golikov

IPC: G10L15/22 ,  G10L15/18 ,  G10L15/05 ,  G10L25/78

Abstract: Some implementations process, using warm word model(s), a stream of audio data to determine a portion of the audio data that corresponds to particular word(s) and/or phrase(s) (e.g., a warm word) associated with an assistant command, process, using an automatic speech recognition (ASR) model, a preamble portion of the audio data (e.g., that precedes the warm word) and/or a postamble portion of the audio data (e.g., that follows the warm word) to generate ASR output, and determine, based on processing the ASR output, whether a user intended the assistant command to be performed. Additional or alternative implementations can process the stream of audio data using a speaker identification (SID) model to determine whether the audio data is sufficient to identify the user that provided a spoken utterance captured in the stream of audio data, and determine if that user is authorized to cause performance of the assistant command.