公开(公告)号：US12073824B2

公开(公告)日：2024-08-27

申请号：US17616135

申请日：2020-12-03

Applicant: GOOGLE LLC

Inventor： Tara N. Sainath ,  Yanzhang He ,  Bo Li ,  Arun Narayanan ,  Ruoming Pang ,  Antoine Jean Bruguier ,  Shuo-Yiin Chang ,  Wei Li

IPC: G10L15/00 ,  G06N3/08 ,  G10L15/05 ,  G10L15/06 ,  G10L15/16 ,  G10L15/22

CPC classification number: G10L15/16 ,  G06N3/08 ,  G10L15/05 ,  G10L15/063 ,  G10L15/22 ,  G10L2015/0635

Abstract: Two-pass automatic speech recognition (ASR) models can be used to perform streaming on-device ASR to generate a text representation of an utterance captured in audio data. Various implementations include a first-pass portion of the ASR model used to generate streaming candidate recognition(s) of an utterance captured in audio data. For example, the first-pass portion can include a recurrent neural network transformer (RNN-T) decoder. Various implementations include a second-pass portion of the ASR model used to revise the streaming candidate recognition(s) of the utterance and generate a text representation of the utterance. For example, the second-pass portion can include a listen attend spell (LAS) decoder. Various implementations include a shared encoder shared between the RNN-T decoder and the LAS decoder.

公开(公告)号：US11676625B2

公开(公告)日：2023-06-13

申请号：US17152918

申请日：2021-01-20

Applicant: Google LLC

Inventor： Shuo-Yiin Chang ,  Bo Li ,  Gabor Simko ,  Maria Carolina Parada San Martin ,  Sean Matthew Shannon

IPC: G10L15/16 ,  G10L25/78 ,  G06N3/08 ,  G06N20/20 ,  G06N5/046 ,  G06F18/214 ,  G06N3/045

CPC classification number: G10L25/78 ,  G06F18/214 ,  G06N3/045 ,  G06N3/08 ,  G06N5/046 ,  G06N20/20 ,  G10L15/16

Abstract: A method for training an endpointer model includes short-form speech utterances and long-form speech utterances. The method also includes providing a short-form speech utterance as input to a shared neural network, the shared neural network configured to learn shared hidden representations suitable for both voice activity detection (VAD) and end-of-query (EOQ) detection. The method also includes generating, using a VAD classifier, a sequence of predicted VAD labels and determining a VAD loss by comparing the sequence of predicted VAD labels to a corresponding sequence of reference VAD labels. The method also includes, generating, using an EOQ classifier, a sequence of predicted EOQ labels and determining an EOQ loss by comparing the sequence of predicted EOQ labels to a corresponding sequence of reference EOQ labels. The method also includes training, using a cross-entropy criterion, the endpointer model based on the VAD loss and the EOQ loss.