公开(公告)号：US11610586B2

公开(公告)日：2023-03-21

申请号：US17182592

申请日：2021-02-23

Applicant: Google LLC

Inventor： David Qiu ,  Qiujia Li ,  Yanzhang He ,  Yu Zhang ,  Bo Li ,  Liangliang Cao ,  Rohit Prabhavalkar ,  Deepti Bhatia ,  Wei Li ,  Ke Hu ,  Tara Sainath ,  Ian Mcgraw

IPC: G10L15/22 ,  G10L15/08 ,  G06N3/08 ,  G10L25/30

Abstract: A method includes receiving a speech recognition result, and using a confidence estimation module (CEM), for each sub-word unit in a sequence of hypothesized sub-word units for the speech recognition result: obtaining a respective confidence embedding that represents a set of confidence features; generating, using a first attention mechanism, a confidence feature vector; generating, using a second attention mechanism, an acoustic context vector; and generating, as output from an output layer of the CEM, a respective confidence output score for each corresponding sub-word unit based on the confidence feature vector and the acoustic feature vector received as input by the output layer of the CEM. For each of the one or more words formed by the sequence of hypothesized sub-word units, the method also includes determining a respective word-level confidence score for the word. The method also includes determining an utterance-level confidence score by aggregating the word-level confidence scores.

公开(公告)号：US20200349923A1

公开(公告)日：2020-11-05

申请号：US16861190

申请日：2020-04-28

Applicant: Google LLC

Inventor： Ke Hu ,  Antoine Jean Bruguier ,  Tara N. Sainath ,  Rohit Prakash Prabhavalkar ,  Golan Pundak

IPC: G10L15/06 ,  G10L15/187 ,  G10L15/193 ,  G10L15/32 ,  G10L15/28 ,  G10L25/30 ,  G10L15/02

Abstract: A method includes receiving audio data encoding an utterance spoken by a native speaker of a first language, and receiving a biasing term list including one or more terms in a second language different than the first language. The method also includes processing, using a speech recognition model, acoustic features derived from the audio data to generate speech recognition scores for both wordpieces and corresponding phoneme sequences in the first language. The method also includes rescoring the speech recognition scores for the phoneme sequences based on the one or more terms in the biasing term list, and executing, using the speech recognition scores for the wordpieces and the rescored speech recognition scores for the phoneme sequences, a decoding graph to generate a transcription for the utterance.

公开(公告)号：US11908461B2

公开(公告)日：2024-02-20

申请号：US17149018

申请日：2021-01-14

Applicant: Google LLC

Inventor： Ke Hu ,  Tara N. Sainath ,  Ruoming Pang ,  Rohit Prakash Prabhavalkar

IPC: G10L15/18 ,  G06N3/049 ,  G10L15/06 ,  G10L15/16 ,  G10L15/187 ,  G10L19/00

CPC classification number: G10L15/1815 ,  G06N3/049 ,  G10L15/063 ,  G10L15/16 ,  G10L15/187 ,  G10L19/0018

Abstract: A method of performing speech recognition using a two-pass deliberation architecture includes receiving a first-pass hypothesis and an encoded acoustic frame and encoding the first-pass hypothesis at a hypothesis encoder. The first-pass hypothesis is generated by a recurrent neural network (RNN) decoder model for the encoded acoustic frame. The method also includes generating, using a first attention mechanism attending to the encoded acoustic frame, a first context vector, and generating, using a second attention mechanism attending to the encoded first-pass hypothesis, a second context vector. The method also includes decoding the first context vector and the second context vector at a context vector decoder to form a second-pass hypothesis.

公开(公告)号：US20220172706A1

公开(公告)日：2022-06-02

申请号：US17651315

申请日：2022-02-16

Applicant: Google LLC

Inventor： Ke Hu ,  Golan Pundak ,  Rohit Prakash Prabhavalkar ,  Antoine Jean Bruguier ,  Tara N. Sainath

IPC: G10L15/06 ,  G10L15/02 ,  G10L15/187 ,  G10L15/193 ,  G10L15/28 ,  G10L15/32 ,  G10L25/30

Abstract: A method includes receiving audio data encoding an utterance spoken by a native speaker of a first language, and receiving a biasing term list including one or more terms in a second language different than the first language. The method also includes processing, using a speech recognition model, acoustic features derived from the audio data to generate speech recognition scores for both wordpieces and corresponding phoneme sequences in the first language. The method also includes rescoring the speech recognition scores for the phoneme sequences based on the one or more terms in the biasing term list, and executing, using the speech recognition scores for the wordpieces and the rescored speech recognition scores for the phoneme sequences, a decoding graph to generate a transcription for the utterance.

公开(公告)号：US20210225369A1

公开(公告)日：2021-07-22

申请号：US17149018

申请日：2021-01-14

Applicant: Google LLC

Inventor： Ke Hu ,  Tara N. Sainath ,  Ruoming Pang ,  Rohit Prakash Prabhavalkar

IPC: G10L15/18 ,  G10L15/06 ,  G06N3/04 ,  G10L15/187 ,  G10L15/16 ,  G10L19/00

Abstract: A method of performing speech recognition using a two-pass deliberation architecture includes receiving a first-pass hypothesis and an encoded acoustic frame and encoding the first-pass hypothesis at a hypothesis encoder. The first-pass hypothesis is generated by a recurrent neural network (RNN) decoder model for the encoded acoustic frame. The method also includes generating, using a first attention mechanism attending to the encoded acoustic frame, a first context vector, and generating, using a second attention mechanism attending to the encoded first-pass hypothesis, a second context vector. The method also includes decoding the first context vector and the second context vector at a context vector decoder to form a second-pass hypothesis