公开(公告)号：US20240203406A1

公开(公告)日：2024-06-20

申请号：US18065685

申请日：2022-12-14

Applicant: Google LLC

Inventor： Soheil Khorram ,  Anshuman Tripathi ,  Kim Jaeyoung ,  Han Lu ,  Qian Zhang ,  Hasim Sak

IPC: G10L15/183 ,  G10L15/06 ,  G10L15/22

CPC classification number: G10L15/183 ,  G10L15/063 ,  G10L15/22

Abstract: A method includes receiving a sequence of acoustic frames extracted from unlabeled audio samples that correspond to spoken utterances not paired with any corresponding transcriptions. The method also includes generating, using a supervised audio encoder, a target higher order feature representation for a corresponding acoustic frame. The method also includes augmenting the sequence of acoustic frames and generating, as output form an unsupervised audio encoder, a predicted higher order feature representation for a corresponding augmented acoustic frame in the sequence of augmented acoustic frames. The method also includes determining an unsupervised loss term based on the target higher order feature representation and the predicted higher order feature representation and updating parameters of the speech recognition model based on the unsupervised loss term.

公开(公告)号：US11961515B2

公开(公告)日：2024-04-16

申请号：US17644337

申请日：2021-12-14

Applicant: Google LLC

Inventor： Jaeyoung Kim ,  Soheil Khorram ,  Hasim Sak ,  Anshuman Tripathi ,  Han Lu ,  Qian Zhang

IPC: G10L15/16 ,  G06N3/088 ,  G10L15/18

CPC classification number: G10L15/16 ,  G06N3/088 ,  G10L15/1815

Abstract: A method includes receiving a plurality of unlabeled audio samples corresponding to spoken utterances not paired with corresponding transcriptions. At a target branch of a contrastive Siamese network, the method also includes generating a sequence of encoder outputs for the plurality of unlabeled audio samples and modifying time characteristics of the encoder outputs to generate a sequence of target branch outputs. At an augmentation branch of a contrastive Siamese network, the method also includes performing augmentation on the unlabeled audio samples, generating a sequence of augmented encoder outputs for the augmented unlabeled audio samples, and generating predictions of the sequence of target branch outputs generated at the target branch. The method also includes determining an unsupervised loss term based on target branch outputs and predictions of the sequence of target branch outputs. The method also includes updating parameters of the audio encoder based on the unsupervised loss term.

公开(公告)号：US20240290322A1

公开(公告)日：2024-08-29

申请号：US18587860

申请日：2024-02-26

Applicant: Google LLC

Inventor： JAEYOUNG Kim ,  Han Lu ,  Soheil Khorram ,  Anshuman Tripathi ,  Qian Zhang ,  Hasim Sak

IPC: G10L15/06

CPC classification number: G10L15/063

Abstract: A method of training an accent recognition model includes receiving a corpus of training utterances spoken across various accents, each training utterance in the corpus including training audio features characterizing the training utterance, and executing a training process to train the accent recognition model on the corpus of training utterances to teach the accent recognition model to learn how to predict accent representations from the training audio features. The accent recognition model includes one or more strided convolution layers, a stack of multi-headed attention layers, and a pooling layer configured to generate a corresponding accent representation.

公开(公告)号：US20240242712A1

公开(公告)日：2024-07-18

申请号：US18619684

申请日：2024-03-28

Applicant: Google LLC

Inventor： Jaeyoung Kim ,  Soheil Khorram ,  Hasim Sak ,  Anshuman Tripathi ,  Han Lu ,  Qian Zhang

IPC: G10L15/16 ,  G06N3/088 ,  G10L15/18

CPC classification number: G10L15/16 ,  G06N3/088 ,  G10L15/1815

Abstract: A method includes receiving a plurality of unlabeled audio samples corresponding to spoken utterances not paired with corresponding transcriptions. At a target branch of a contrastive Siamese network, the method also includes generating a sequence of encoder outputs for the plurality of unlabeled audio samples and modifying time characteristics of the encoder outputs to generate a sequence of target branch outputs. At an augmentation branch of a contrastive Siamese network, the method also includes performing augmentation on the unlabeled audio samples, generating a sequence of augmented encoder outputs for the augmented unlabeled audio samples, and generating predictions of the sequence of target branch outputs generated at the target branch. The method also includes determining an unsupervised loss term based on target branch outputs and predictions of the sequence of target branch outputs. The method also includes updating parameters of the audio encoder based on the unsupervised loss term.

公开(公告)号：US20240177706A1

公开(公告)日：2024-05-30

申请号：US18515212

申请日：2023-11-20

Applicant: Google LLC

Inventor： Anshuman Tripathi ,  Soheil Khorram ,  Hasim Sak ,  Han Lu ,  Jaeyoung Kim ,  Qian Zhang

IPC: G10L15/06 ,  G10L15/065 ,  G10L15/10

CPC classification number: G10L15/063 ,  G10L15/065 ,  G10L15/10 ,  G10L2015/0635

Abstract: A method for training a sequence transduction model includes receiving a sequence of unlabeled input features extracted from unlabeled input samples. Using a teacher branch of an unsupervised subnetwork, the method includes processing the sequence of input features to predict probability distributions over possible teacher branch output labels, sampling one or more sequences of teacher branch output labels, and determining a sequence of pseudo output labels based on the one or more sequences of teacher branch output labels. Using a student branch that includes a student encoder of the unsupervised subnetwork, the method includes processing the sequence of input 10 features to predict probability distributions over possible student branch output labels, determining a negative log likelihood term based on the predicted probability distributions over possible student branch output labels and the sequence of pseudo output labels, and updating parameters of the student encoder.

公开(公告)号：US20230096805A1

公开(公告)日：2023-03-30

申请号：US17644337

申请日：2021-12-14

Applicant: Google LLC

Inventor： Jaeyoung Kim ,  Soheil Khorram ,  Hasim Sak ,  Anshuman Tripathi ,  Han Lu ,  Qian Zhang

IPC: G10L15/16 ,  G10L15/18 ,  G06N3/08

Abstract: A method includes receiving a plurality of unlabeled audio samples corresponding to spoken utterances not paired with corresponding transcriptions. At a target branch of a contrastive Siamese network, the method also includes generating a sequence of encoder outputs for the plurality of unlabeled audio samples and modifying time characteristics of the encoder outputs to generate a sequence of target branch outputs. At an augmentation branch of a contrastive Siamese network, the method also includes performing augmentation on the unlabeled audio samples, generating a sequence of augmented encoder outputs for the augmented unlabeled audio samples, and generating predictions of the sequence of target branch outputs generated at the target branch. The method also includes determining an unsupervised loss term based on target branch outputs and predictions of the sequence of target branch outputs. The method also includes updating parameters of the audio encoder based on the unsupervised loss term.