公开(公告)号：US10679615B2

公开(公告)日：2020-06-09

申请号：US15973461

申请日：2018-05-07

Applicant: Google LLC

Inventor： Pu-sen Chao ,  Diego Melendo Casado ,  Ignacio Lopez Moreno

IPC: G10L15/00 ,  G10L15/18 ,  G10L15/08 ,  G10L15/14 ,  G10L15/22 ,  G10L15/30

Abstract: Determining a language for speech recognition of a spoken utterance received via an automated assistant interface for interacting with an automated assistant. The system can enable multilingual interaction with the automated assistant, without necessitating a user explicitly designate a language to be utilized for each interaction. The system can determine a user profile that corresponds to audio data that captures a spoken utterance, and utilize language(s), and optionally corresponding probabilities, assigned to the user profile in determining a language for speech recognition of the spoken utterance. The system can perform speech recognition in each of multiple languages assigned to the user profile, and utilize criteria to select only one of the speech recognitions as appropriate for generating and providing content that is responsive to the spoken utterance.

公开(公告)号：US20200160869A1

公开(公告)日：2020-05-21

申请号：US16752007

申请日：2020-01-24

Applicant: Google LLC

Inventor： Georg Heigold ,  Samuel Bengio ,  Ignacio Lopez Moreno

IPC: G10L17/18 ,  G10L17/02 ,  G10L17/04

Abstract: This document generally describes systems, methods, devices, and other techniques related to speaker verification, including (i) training a neural network for a speaker verification model, (ii) enrolling users at a client device, and (iii) verifying identities of users based on characteristics of the users' voices. Some implementations include a computer-implemented method. The method can include receiving, at a computing device, data that characterizes an utterance of a user of the computing device. A speaker representation can be generated, at the computing device, for the utterance using a neural network on the computing device. The neural network can be trained based on a plurality of training samples that each: (i) include data that characterizes a first utterance and data that characterizes one or more second utterances, and (ii) are labeled as a matching speakers sample or a non-matching speakers sample.

公开(公告)号：US20200152207A1

公开(公告)日：2020-05-14

申请号：US16617219

申请日：2019-04-15

Applicant: Google LLC

Inventor： Quan Wang ,  Yash Sheth ,  Ignacio Lopez Moreno ,  Li Wan

IPC: G10L17/18 ,  G10L17/04 ,  G10L15/26

Abstract: Techniques are described for training and/or utilizing an end-to-end speaker diarization model. In various implementations, the model is a recurrent neural network (RNN) model, such as an RNN model that includes at least one memory layer, such as a long short-term memory (LSTM) layer. Audio features of audio data can be applied as input to an end-to-end speaker diarization model trained according to implementations disclosed herein, and the model utilized to process the audio features to generate, as direct output over the model, speaker diarization results. Further, the end-to-end speaker diarization model can be a sequence-to-sequence model, where the sequence can have variable length. Accordingly, the model can be utilized to generate speaker diarization results for any of various length audio segments.

公开(公告)号：US10565996B2

公开(公告)日：2020-02-18

申请号：US15170264

申请日：2016-06-01

Applicant: Google LLC

Inventor： Matthew Sharifi ,  Ignacio Lopez Moreno ,  Ludwig Schmidt

IPC: G10L17/02 ,  G10L17/00 ,  G10L17/08 ,  G10L17/18 ,  G10L25/51

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing speaker identification. In some implementations, data identifying a media item including speech of a speaker is received. Based on the received data, one or more other media items that include speech of the speaker are identified. One or more search results are generated that each reference a respective media item of the one or more other media items that include speech of the speaker. The one or more search results are provided for display.

公开(公告)号：US20200051553A1

公开(公告)日：2020-02-13

申请号：US16548947

申请日：2019-08-23

Applicant: Google LLC

Inventor： Diego Melendo Casado ,  Ignacio Lopez Moreno ,  Javier Gonzalez-Dominguez

IPC: G10L15/20 ,  G10L25/84 ,  G10L21/034 ,  G10L17/06 ,  G10L15/22 ,  H03G3/30 ,  G06F3/16

Abstract: The technology described in this document can be embodied in a computer-implemented method that includes receiving, at a processing system, a first signal including an output of a speaker device and an additional audio signal. The method also includes determining, by the processing system, based at least in part on a model trained to identify the output of the speaker device, that the additional audio signal corresponds to an utterance of a user. The method further includes initiating a reduction in an audio output level of the speaker device based on determining that the additional audio signal corresponds to the utterance of the user.

公开(公告)号：US20190318735A1

公开(公告)日：2019-10-17

申请号：US16163327

申请日：2018-10-17

Applicant: Google LLC

Inventor： Pu-sen Chao ,  Diego Melendo Casado ,  Ignacio Lopez Moreno ,  William Zhang

IPC: G10L15/22 ,  G10L15/14 ,  G10L15/00

Abstract: Determining a language for speech recognition of a spoken utterance received via an automated assistant interface for interacting with an automated assistant. Implementations can enable multilingual interaction with the automated assistant, without necessitating a user explicitly designate a language to be utilized for each interaction. Implementations determine a user profile that corresponds to audio data that captures a spoken utterance, and utilize language(s), and optionally corresponding probabilities, assigned to the user profile in determining a language for speech recognition of the spoken utterance. Some implementations select only a subset of languages, assigned to the user profile, to utilize in speech recognition of a given spoken utterance of the user. Some implementations perform speech recognition in each of multiple languages assigned to the user profile, and utilize criteria to select only one of the speech recognitions as appropriate for generating and providing content that is responsive to the spoken utterance.

公开(公告)号：US20180277124A1

公开(公告)日：2018-09-27

申请号：US15995480

申请日：2018-06-01

Applicant: Google LLC

Inventor： Ignacio Lopez Moreno ,  Li Wan ,  Quan Wang

IPC: G10L17/24 ,  G10L17/18 ,  G10L17/02 ,  G10L17/08 ,  G10L17/14

CPC classification number: G10L17/24 ,  G10L17/02 ,  G10L17/08 ,  G10L17/14 ,  G10L17/18 ,  G10L17/22

Abstract: Methods, systems, apparatus, including computer programs encoded on computer storage medium, to facilitate language independent-speaker verification. In one aspect, a method includes actions of receiving, by a user device, audio data representing an utterance of a user. Other actions may include providing, to a neural network stored on the user device, input data derived from the audio data and a language identifier. The neural network may be trained using speech data representing speech in different languages or dialects. The method may include additional actions of generating, based on output of the neural network, a speaker representation and determining, based on the speaker representation and a second representation, that the utterance is an utterance of the user. The method may provide the user with access to the user device based on determining that the utterance is an utterance of the user.

公开(公告)号：US12175963B2

公开(公告)日：2024-12-24

申请号：US18525475

申请日：2023-11-30

Applicant: Google LLC

Inventor： Ye Jia ,  Zhifeng Chen ,  Yonghui Wu ,  Jonathan Shen ,  Ruoming Pang ,  Ron J. Weiss ,  Ignacio Lopez Moreno ,  Fei Ren ,  Yu Zhang ,  Quan Wang ,  Patrick An Phu Nguyen

IPC: G10L13/04 ,  G10L17/04 ,  G10L19/00 ,  G06N3/08 ,  G10L13/02

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for speech synthesis. The methods, systems, and apparatus include actions of obtaining an audio representation of speech of a target speaker, obtaining input text for which speech is to be synthesized in a voice of the target speaker, generating a speaker vector by providing the audio representation to a speaker encoder engine that is trained to distinguish speakers from one another, generating an audio representation of the input text spoken in the voice of the target speaker by providing the input text and the speaker vector to a spectrogram generation engine that is trained using voices of reference speakers to generate audio representations, and providing the audio representation of the input text spoken in the voice of the target speaker for output.

公开(公告)号：US12125476B2

公开(公告)日：2024-10-22

申请号：US17652801

申请日：2022-02-28

Applicant: Google LLC

Inventor： Hyun Jin Park ,  Alex Seungryong Park ,  Ignacio Lopez Moreno

IPC: G10L15/16 ,  G06N3/08 ,  G10L15/02 ,  G10L15/06 ,  G10L15/22 ,  G06N3/0455 ,  G10L15/08

CPC classification number: G10L15/16 ,  G06N3/08 ,  G10L15/02 ,  G10L15/063 ,  G10L15/22 ,  G06N3/0455 ,  G10L2015/025 ,  G10L2015/088

Abstract: A method for training a neural network includes receiving a training input audio sequence including a sequence of input frames defining a hotword that initiates a wake-up process on a user device. The method further includes obtaining a first label and a second label for the training input audio sequence. The method includes generating, using a memorized neural network and the training input audio sequence, an output indicating a likelihood the training input audio sequence includes the hotword. The method further includes determining a first loss based on the first label and the output. The method includes determining a second loss based on the second label and the output. The method further includes optimizing the memorized neural network based on the first loss and the second loss associated with the training input audio sequence.

公开(公告)号：US12027162B2

公开(公告)日：2024-07-02

申请号：US17190779

申请日：2021-03-03

Applicant: GOOGLE LLC

Inventor： Hyun Jin Park ,  Pai Zhu ,  Ignacio Lopez Moreno ,  Niranjan Subrahmanya

IPC: G10L15/22 ,  G06F18/24 ,  G10L15/06 ,  G10L15/08 ,  G10L21/0208

CPC classification number: G10L15/22 ,  G06F18/24 ,  G10L15/063 ,  G10L15/08 ,  G10L21/0208 ,  G10L2015/088 ,  G10L2015/223 ,  G10L2021/02082 ,  G10L2021/02087

Abstract: Teacher-student learning can be used to train a keyword spotting (KWS) model using augmented training instance(s). Various implementations include aggressively augmenting (e.g., using spectral augmentation) base audio data to generate augmented audio data, where one or more portions of the base instance of audio data can be masked in the augmented instance of audio data (e.g., one or more time frames can be masked, one or more frequencies can be masked, etc.). Many implementations include processing augmented audio data using a KWS teacher model to generate a soft label, and processing the augmented audio data using a KWS student model to generate predicted output. One or more portions of the KWS student model can be updated based on a comparison of the soft label and the generated predicted output.