公开(公告)号：US20250078840A1

公开(公告)日：2025-03-06

申请号：US18812338

申请日：2024-08-22

Applicant: Google LLC

Inventor： Pai Zhu ,  Beltrán Labrador Serrano ,  Guanlong Zhao ,  Angelo Alfredo Scorza Scarpati ,  Quan Wang ,  Alex Seungryong Park ,  Ignacio Lopez Moreno

IPC: G10L17/02 ,  G10L17/04 ,  G10L17/22

Abstract: A method includes receiving audio data corresponding to an utterance spoken by a particular user and captured in streaming audio by a user device. The method also includes performing speaker identification on the audio data to identify an identity of the particular user that spoke the utterance. The method also includes obtaining a keyword detection model personalized for the particular user based on the identity of the particular user that spoke the utterance. The keyword detection model is conditioned on speaker characteristic information associated with the particular user to adapt the keyword detection model to detect a presence of a keyword in audio for the particular user. The method also includes determining that the utterance includes the keyword using the keyword detection model personalized for the particular user.

公开(公告)号：US12159622B2

公开(公告)日：2024-12-03

申请号：US18078476

申请日：2022-12-09

Applicant: GOOGLE LLC

Inventor： Pu-sen Chao ,  Diego Melendo Casado ,  Ignacio Lopez Moreno ,  Quan Wang

IPC: G10L15/22 ,  G10L15/06 ,  G10L15/07 ,  G10L15/32 ,  G10L17/24

Abstract: Text independent speaker recognition models can be utilized by an automated assistant to verify a particular user spoke a spoken utterance and/or to identify the user who spoke a spoken utterance. Implementations can include automatically updating a speaker embedding for a particular user based on previous utterances by the particular user. Additionally or alternatively, implementations can include verifying a particular user spoke a spoken utterance using output generated by both a text independent speaker recognition model as well as a text dependent speaker recognition model. Furthermore, implementations can additionally or alternatively include prefetching content for several users associated with a spoken utterance prior to determining which user spoke the spoken utterance.

公开(公告)号：US20240112667A1

公开(公告)日：2024-04-04

申请号：US18525475

申请日：2023-11-30

Applicant: Google LLC

Inventor： Ye Jia ,  Zhifeng Chen ,  Yonghui Wu ,  Jonathan Shen ,  Ruoming Pang ,  Ron J. Weiss ,  Ignacio Lopez Moreno ,  Fei Ren ,  Yu Zhang ,  Quan Wang ,  Patrick An Phu Nguyen

IPC: G10L13/04 ,  G10L17/04 ,  G10L19/00

CPC classification number: G10L13/04 ,  G10L17/04 ,  G10L19/00 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for speech synthesis. The methods, systems, and apparatus include actions of obtaining an audio representation of speech of a target speaker, obtaining input text for which speech is to be synthesized in a voice of the target speaker, generating a speaker vector by providing the audio representation to a speaker encoder engine that is trained to distinguish speakers from one another, generating an audio representation of the input text spoken in the voice of the target speaker by providing the input text and the speaker vector to a spectrogram generation engine that is trained using voices of reference speakers to generate audio representations, and providing the audio representation of the input text spoken in the voice of the target speaker for output.

公开(公告)号：US11942094B2

公开(公告)日：2024-03-26

申请号：US17211791

申请日：2021-03-24

Applicant: Google LLC

Inventor： Roza Chojnacka ,  Jason Pelecanos ,  Quan Wang ,  Ignacio Lopez Moreno

IPC: G10L17/02 ,  G06F16/9032 ,  G10L15/08

CPC classification number: G10L17/02 ,  G06F16/90332 ,  G10L2015/088

Abstract: A speaker verification method includes receiving audio data corresponding to an utterance, processing a first portion of the audio data that characterizes a predetermined hotword to generate a text-dependent evaluation vector, and generating one or more text-dependent confidence scores. When one of the text-dependent confidence scores satisfies a threshold, the operations include identifying a speaker of the utterance as a respective enrolled user associated with the text-dependent confidence score that satisfies the threshold and initiating performance of an action without performing speaker verification. When none of the text-dependent confidence scores satisfy the threshold, the operations include processing a second portion of the audio data that characterizes a query to generate a text-independent evaluation vector, generating one or more text-independent confidence scores, and determining whether the identity of the speaker of the utterance includes any of the enrolled users.

公开(公告)号：US11922951B2

公开(公告)日：2024-03-05

申请号：US17567590

申请日：2022-01-03

Applicant: GOOGLE LLC

Inventor： Quan Wang ,  Prashant Sridhar ,  Ignacio Lopez Moreno ,  Hannah Muckenhirn

IPC: G10L17/04 ,  G10L17/00 ,  G10L17/02 ,  G10L17/18 ,  G10L17/22 ,  G10L25/18

CPC classification number: G10L17/04 ,  G10L17/00 ,  G10L17/02 ,  G10L17/18 ,  G10L17/22 ,  G10L25/18

Abstract: Techniques are disclosed that enable processing of audio data to generate one or more refined versions of audio data, where each of the refined versions of audio data isolate one or more utterances of a single respective human speaker. Various implementations generate a refined version of audio data that isolates utterance(s) of a single human speaker by processing a spectrogram representation of the audio data (generated by processing the audio data with a frequency transformation) using a mask generated by processing the spectrogram of the audio data and a speaker embedding for the single human speaker using a trained voice filter model. Output generated over the trained voice filter model is processed using an inverse of the frequency transformation to generate the refined audio data.

公开(公告)号：US20230274731A1

公开(公告)日：2023-08-31

申请号：US17652801

申请日：2022-02-28

Applicant: Google LLC

Inventor： Hyun Jin Park ,  Alex Seungryong Park ,  Ignacio Lopez Moreno

IPC: G10L15/16 ,  G10L15/06 ,  G10L15/22 ,  G10L15/02 ,  G06N3/08

CPC classification number: G10L15/16 ,  G10L15/063 ,  G10L15/22 ,  G10L15/02 ,  G06N3/08 ,  G10L2015/088

Abstract: A method for training a neural network includes receiving a training input audio sequence including a sequence of input frames defining a hotword that initiates a wake-up process on a user device. The method further includes obtaining a first label and a second label for the training input audio sequence. The method includes generating, using a memorized neural network and the training input audio sequence, an output indicating a likelihood the training input audio sequence includes the hotword. The method further includes determining a first loss based on the first label and the output. The method includes determining a second loss based on the second label and the output. The method further includes optimizing the memorized neural network based on the first loss and the second loss associated with the training input audio sequence.

公开(公告)号：US11735173B2

公开(公告)日：2023-08-22

申请号：US17328400

申请日：2021-05-24

Applicant: Google LLC

Inventor： Pu-sen Chao ,  Diego Melendo Casado ,  Ignacio Lopez Moreno ,  William Zhang

IPC: G10L15/22 ,  G10L15/197 ,  G10L15/00 ,  G10L15/30 ,  G10L15/08 ,  G10L15/14 ,  G10L15/18 ,  G10L13/00

CPC classification number: G10L15/197 ,  G10L13/00 ,  G10L15/005 ,  G10L15/08 ,  G10L15/14 ,  G10L15/1822 ,  G10L15/22 ,  G10L15/30 ,  G10L2015/088 ,  G10L2015/223 ,  G10L2015/228

Abstract: Determining a language for speech recognition of a spoken utterance received via an automated assistant interface for interacting with an automated assistant. Implementations can enable multilingual interaction with the automated assistant, without necessitating a user explicitly designate a language to be utilized for each interaction. Implementations determine a user profile that corresponds to audio data that captures a spoken utterance, and utilize language(s), and optionally corresponding probabilities, assigned to the user profile in determining a language for speech recognition of the spoken utterance. Some implementations select only a subset of languages, assigned to the user profile, to utilize in speech recognition of a given spoken utterance of the user. Some implementations perform speech recognition in each of multiple languages assigned to the user profile, and utilize criteria to select only one of the speech recognitions as appropriate for generating and providing content that is responsive to the spoken utterance.

公开(公告)号：US11620989B2

公开(公告)日：2023-04-04

申请号：US16452959

申请日：2019-06-26

Applicant: Google LLC

Inventor： Ignacio Lopez Moreno ,  Yu-hsin Joyce Chen

IPC: G10L15/16 ,  G06N3/04 ,  G10L15/08 ,  G10L17/18

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network. One of the methods includes generating, by a speech recognition system, a matrix from a predetermined quantity of vectors that each represent input for a layer of a neural network, generating a plurality of sub-matrices from the matrix, using, for each of the sub-matrices, the respective sub-matrix as input to a node in the layer of the neural network to determine whether an utterance encoded in an audio signal comprises a keyword for which the neural network is trained.

公开(公告)号：US20230089308A1

公开(公告)日：2023-03-23

申请号：US17644261

申请日：2021-12-14

Applicant: Google LLC

Inventor： Quan Wang ,  Han Lu ,  Evan Clark ,  Ignacio Lopez Moreno ,  Hasim Sak ,  Wei Xia ,  Taral Joglekar ,  Anshuman Tripathi

IPC: G10L15/26 ,  G10L15/16 ,  G10L15/06

Abstract: A method includes receiving an input audio signal that corresponds to utterances spoken by multiple speakers. The method also includes processing the input audio to generate a transcription of the utterances and a sequence of speaker turn tokens each indicating a location of a respective speaker turn. The method also includes segmenting the input audio signal into a plurality of speaker segments based on the sequence of speaker tokens. The method also includes extracting a speaker-discriminative embedding from each speaker segment and performing spectral clustering on the speaker-discriminative embeddings to cluster the plurality of speaker segments into k classes. The method also includes assigning a respective speaker label to each speaker segment clustered into the respective class that is different than the respective speaker label assigned to the speaker segments clustered into each other class of the k classes.

公开(公告)号：US20220366914A1

公开(公告)日：2022-11-17

申请号：US17302926

申请日：2021-05-16

Applicant: Google LLC

Inventor： Ignacio Lopez Moreno ,  Quan Wang ,  Jason Pelecanos ,  Yiling Huang ,  Mert Saglam

IPC: G10L17/06 ,  G10L17/18 ,  G10L17/04 ,  G06F16/245 ,  G06N3/08

Abstract: A speaker verification method includes receiving audio data corresponding to an utterance, processing the audio data to generate a reference attentive d-vector representing voice characteristics of the utterance, the evaluation ad-vector includes ne style classes each including a respective value vector concatenated with a corresponding routing vector. The method also includes generating using a self-attention mechanism, at least one multi-condition attention score that indicates a likelihood that the evaluation ad-vector matches a respective reference ad-vector associated with a respective user. The method also includes identifying the speaker of the utterance as the respective user associated with the respective reference ad-vector based on the multi-condition attention score.