公开(公告)号：US20240420692A1

公开(公告)日：2024-12-19

申请号：US18818010

申请日：2024-08-28

Applicant: Google LLC

Inventor： Neeraj Gaur ,  Tongzhou Chen ,  Ehsan Variani ,  Bhuvana Ramabhadran ,  Parisa Haghani ,  Pedro J. Moreno Mengibar

IPC: G10L15/197 ,  G10L15/00 ,  G10L15/16 ,  G10L15/22

Abstract: A method includes receiving a sequence of acoustic frames extracted from audio data corresponding to an utterance. During a first pass, the method includes processing the sequence of acoustic frames to generate N candidate hypotheses for the utterance. During a second pass, and for each candidate hypothesis, the method includes: generating a respective un-normalized likelihood score; generating a respective external language model score; generating a standalone score that models prior statistics of the corresponding candidate hypothesis; and generating a respective overall score for the candidate hypothesis based on the un-normalized likelihood score, the external language model score, and the standalone score. The method also includes selecting the candidate hypothesis having the highest respective overall score from among the N candidate hypotheses as a final transcription of the utterance.

公开(公告)号：US20240304178A1

公开(公告)日：2024-09-12

申请号：US18439630

申请日：2024-02-12

Applicant: Google LLC

Inventor： Andrew M Rosenberg ,  Yacob Yochai Blau ,  Bhuvana Ramabhadran ,  Genady Beryozkin ,  Gary Wang ,  Zhehuai Chen ,  Rohan Agrawal ,  Parisa Haghani

IPC: G10L15/06 ,  G10L15/22 ,  G10L15/26

CPC classification number: G10L15/063 ,  G10L15/22 ,  G10L15/26

Abstract: A method includes receiving training data including transcribed speech utterances spoken in a general domain, modified speech utterances in a target domain, and unspoken textual utterances corresponding to the transcriptions of the modified speech utterances in the target domain. The modified speech utterances include utterances spoken in the target domain that have been modified to obfuscate one or more classes of sensitive information recited in the utterances. The method also includes generating a corresponding alignment output for each unspoken textual utterance of the received training data using an alignment model. The method also includes training a speech recognition model on the alignment outputs generated for the corresponding to the unspoken textual utterances, the un-transcribed speech utterances, and the transcribed speech utterances to teach the speech recognition model to learn to recognize speech in the target domain and phrases within the one or more classes of sensitive information.

公开(公告)号：US20250095634A1

公开(公告)日：2025-03-20

申请号：US18965193

申请日：2024-12-02

Applicant: Google LLC

Inventor： Bo Li ,  Tara N. Sainath ,  Ruoming Pang ,  Shuo-yiin Chang ,  Qiumin Xu ,  Trevor Strohman ,  Vince Chen ,  Qiao Liang ,  Heguang Liu ,  Yanzhang He ,  Parisa Haghani ,  Sameer Bidichandani

IPC: G10L15/00 ,  G10L15/06 ,  G10L15/22 ,  G10L15/30

Abstract: A method includes receiving a sequence of acoustic frames characterizing one or more utterances as input to a multilingual automated speech recognition (ASR) model. The method also includes generating a higher order feature representation for a corresponding acoustic frame. The method also includes generating a hidden representation based on a sequence of non-blank symbols output by a final softmax layer. The method also includes generating a probability distribution over possible speech recognition hypotheses based on the hidden representation generated by the prediction network at each of the plurality of output steps and the higher order feature representation generated by the encoder at each of the plurality of output steps. The method also includes predicting an end of utterance (EOU) token at an end of each utterance. The method also includes classifying each acoustic frame as either speech, initial silence, intermediate silence, or final silence.

公开(公告)号：US11741966B2

公开(公告)日：2023-08-29

申请号：US17964141

申请日：2022-10-12

Applicant: GOOGLE LLC

Inventor： Asaf Aharoni ,  Arun Narayanan ,  Nir Shabat ,  Parisa Haghani ,  Galen Tsai Chuang ,  Yaniv Leviathan ,  Neeraj Gaur ,  Pedro J. Moreno Mengibar ,  Rohit Prakash Prabhavalkar ,  Zhongdi Qu ,  Austin Severn Waters ,  Tomer Amiaz ,  Michiel A. U. Bacchiani

IPC: G10L15/26 ,  H04M3/428 ,  H04M1/663 ,  G10L15/32 ,  H04M3/51 ,  H04M1/02

CPC classification number: G10L15/26 ,  G10L15/32 ,  H04M1/02 ,  H04M1/663 ,  H04M3/4286 ,  H04M3/5191

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for an automated calling system are disclosed. In one aspect, a method includes the actions of receiving audio data of an utterance spoken by a user who is having a telephone conversation with a bot. The actions further include determining a context of the telephone conversation. The actions further include determining a user intent of a first previous portion of the telephone conversation spoken by the user and a bot intent of a second previous portion of the telephone conversation outputted by a speech synthesizer of the bot. The actions further include, based on the audio data of the utterance, the context of the telephone conversation, the user intent, and the bot intent, generating synthesized speech of a reply by the bot to the utterance. The actions further include, providing, for output, the synthesized speech.

公开(公告)号：US20220310081A1

公开(公告)日：2022-09-29

申请号：US17701635

申请日：2022-03-22

Applicant: Google LLC

Inventor： Neeraj Gaur ,  Tongzhou Chen ,  Ehsan Variani ,  Bhuvana Ramabhadran ,  Parisa Haghani ,  Pedro J. Moreno Mengibar

IPC: G10L15/197 ,  G10L15/16 ,  G10L15/22 ,  G10L15/00

Abstract: A method includes receiving a sequence of acoustic frames extracted from audio data corresponding to an utterance. During a first pass, the method includes processing the sequence of acoustic frames to generate N candidate hypotheses for the utterance. During a second pass, and for each candidate hypothesis, the method includes generating a respective un-normalized likelihood score; generating a respective external language model score; generating a standalone score that models prior statistics of the corresponding candidate hypothesis, and generating a respective overall score for the candidate hypothesis based on the un-normalized likelihood score, the external language model score, and the standalone score. The method also includes selecting the candidate hypothesis having the highest respective overall score from among the N candidate hypotheses as a final transcription of the utterance.

公开(公告)号：US12183322B2

公开(公告)日：2024-12-31

申请号：US17934555

申请日：2022-09-22

Applicant: Google LLC

Inventor： Bo Li ,  Tara N. Sainath ,  Ruoming Pang ,  Shuo-yiin Chang ,  Qiumin Xu ,  Trevor Strohman ,  Vince Chen ,  Qiao Liang ,  Heguang Liu ,  Yanzhang He ,  Parisa Haghani ,  Sameer Bidichandani

IPC: G10L15/00 ,  G10L15/06 ,  G10L15/22 ,  G10L15/30

Abstract: A method includes receiving a sequence of acoustic frames characterizing one or more utterances as input to a multilingual automated speech recognition (ASR) model. The method also includes generating a higher order feature representation for a corresponding acoustic frame. The method also includes generating a hidden representation based on a sequence of non-blank symbols output by a final softmax layer. The method also includes generating a probability distribution over possible speech recognition hypotheses based on the hidden representation generated by the prediction network at each of the plurality of output steps and the higher order feature representation generated by the encoder at each of the plurality of output steps. The method also includes predicting an end of utterance (EOU) token at an end of each utterance. The method also includes classifying each acoustic frame as either speech, initial silence, intermediate silence, or final silence.

公开(公告)号：US12080283B2

公开(公告)日：2024-09-03

申请号：US17701635

申请日：2022-03-22

Applicant: Google LLC

Inventor： Neeraj Gaur ,  Tongzhou Chen ,  Ehsan Variani ,  Bhuvana Ramabhadran ,  Parisa Haghani ,  Pedro J. Moreno Mengibar

IPC: G10L15/197 ,  G10L15/00 ,  G10L15/16 ,  G10L15/22

CPC classification number: G10L15/197 ,  G10L15/005 ,  G10L15/16 ,  G10L15/22

Abstract: A method includes receiving a sequence of acoustic frames extracted from audio data corresponding to an utterance. During a first pass, the method includes processing the sequence of acoustic frames to generate N candidate hypotheses for the utterance. During a second pass, and for each candidate hypothesis, the method includes: generating a respective un-normalized likelihood score; generating a respective external language model score; generating a standalone score that models prior statistics of the corresponding candidate hypothesis; and generating a respective overall score for the candidate hypothesis based on the un-normalized likelihood score, the external language model score, and the standalone score. The method also includes selecting the candidate hypothesis having the highest respective overall score from among the N candidate hypotheses as a final transcription of the utterance.

公开(公告)号：US20240203409A1

公开(公告)日：2024-06-20

申请号：US18589220

申请日：2024-02-27

Applicant: Google LLC

Inventor： Neeraj Gaur ,  Tongzhou Chen ,  Ehsan Variani ,  Bhuvana Ramabhadran ,  Parisa Haghani ,  Pedro J. Moreno Mengibar

IPC: G10L15/197 ,  G10L15/00 ,  G10L15/16 ,  G10L15/22

CPC classification number: G10L15/197 ,  G10L15/005 ,  G10L15/16 ,  G10L15/22

Abstract: A method includes receiving a sequence of acoustic frames extracted from audio data corresponding to an utterance. During a first pass, the method includes processing the sequence of acoustic frames to generate N candidate hypotheses for the utterance. During a second pass, and for each candidate hypothesis, the method includes: generating a respective un-normalized likelihood score; generating a respective external language model score; generating a standalone score that models prior statistics of the corresponding candidate hypothesis; and generating a respective overall score for the candidate hypothesis based on the un-normalized likelihood score, the external language model score, and the standalone score. The method also includes selecting the candidate hypothesis having the highest respective overall score from among the N candidate hypotheses as a final transcription of the utterance.

公开(公告)号：US20210090570A1

公开(公告)日：2021-03-25

申请号：US16580726

申请日：2019-09-24

Applicant: Google LLC

Inventor： Asaf Aharoni ,  Arun Narayanan ,  Nir Shabat ,  Parisa Haghani ,  Galen Tsai Chuang ,  Yaniv Leviathan ,  Neeraj Gaur ,  Pedro J. Moreno Mengibar ,  Rohit Prakash Prabhavalkar ,  Zhongdi Qu ,  Austin Severn Waters ,  Tomer Amiaz ,  Michiel A.U. Bacchiani

IPC: G10L15/26 ,  H04M3/428 ,  H04M1/02 ,  G10L15/32 ,  H04M3/51 ,  H04M1/663

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for an automated calling system are disclosed. In one aspect, a method includes the actions of receiving audio data of an utterance spoken by a user who is having a telephone conversation with a bot. The actions further include determining a context of the telephone conversation. The actions further include determining a user intent of a first previous portion of the telephone conversation spoken by the user and a bot intent of a second previous portion of the telephone conversation outputted by a speech synthesizer of the bot. The actions further include, based on the audio data of the utterance, the context of the telephone conversation, the user intent, and the bot intent, generating synthesized speech of a reply by the bot to the utterance. The actions further include, providing, for output, the synthesized speech.

公开(公告)号：US12254875B2

公开(公告)日：2025-03-18

申请号：US18589220

申请日：2024-02-27

Applicant: Google LLC

Inventor： Neeraj Gaur ,  Tongzhou Chen ,  Ehsan Variani ,  Bhuvana Ramabhadran ,  Parisa Haghani ,  Pedro J. Moreno Mengibar

IPC: G10L15/197 ,  G10L15/00 ,  G10L15/16 ,  G10L15/22

Abstract: A method includes receiving a sequence of acoustic frames extracted from audio data corresponding to an utterance. During a first pass, the method includes processing the sequence of acoustic frames to generate N candidate hypotheses for the utterance. During a second pass, and for each candidate hypothesis, the method includes: generating a respective un-normalized likelihood score; generating a respective external language model score; generating a standalone score that models prior statistics of the corresponding candidate hypothesis; and generating a respective overall score for the candidate hypothesis based on the un-normalized likelihood score, the external language model score, and the standalone score. The method also includes selecting the candidate hypothesis having the highest respective overall score from among the N candidate hypotheses as a final transcription of the utterance.