公开(公告)号：US20220207321A1

公开(公告)日：2022-06-30

申请号：US17139525

申请日：2020-12-31

Applicant: Google LLC

Inventor： Anmol Gulati ,  Ruoming Pang ,  Niki Parmar ,  Jiahui Yu ,  Wei Han ,  Chung-Cheng Chiu ,  Yu Zhang ,  Yonghui Wu ,  Shibo Wang ,  Weikeng Qin ,  Zhengdong Zhang

IPC: G06N3/04 ,  G10L15/16 ,  G06N20/00

Abstract: Systems and methods can utilize a conformer model to process a data set for various data processing tasks, including, but not limited to, speech recognition, sound separation, protein synthesis determination, video or other image set analysis, and natural language processing. The conformer model can use feed-forward blocks, a self-attention block, and a convolution block to process data to learn global interactions and relative-offset-based local correlations of the input data.

公开(公告)号：US20220101090A1

公开(公告)日：2022-03-31

申请号：US17495398

申请日：2021-10-06

Applicant: Google LLC

Inventor： Mingxing Tan ,  Quoc V. Le ,  Bo Chen ,  Vijay Vasudevan ,  Ruoming Pang

IPC: G06N3/04 ,  G06N20/10 ,  G06F17/15 ,  G06N3/08

Abstract: The present disclosure is directed to an automated neural architecture search approach for designing new neural network architectures such as, for example, resource-constrained mobile CNN models. In particular, the present disclosure provides systems and methods to perform neural architecture search using a novel factorized hierarchical search space that permits layer diversity throughout the network, thereby striking the right balance between flexibility and search space size. The resulting neural architectures are able to be run relatively faster and using relatively fewer computing resources (e.g., less processing power, less memory usage, less power consumption, etc.), all while remaining competitive with or even exceeding the performance (e.g., accuracy) of current state-of-the-art mobile-optimized models.

公开(公告)号：US20210295858A1

公开(公告)日：2021-09-23

申请号：US17222736

申请日：2021-04-05

Applicant: Google LLC

Inventor： Yonghui Wu ,  Jonathan Shen ,  Ruoming Pang ,  Ron J. Weiss ,  Michael Schuster ,  Navdeep Jaitly ,  Zongheng Yang ,  Zhifeng Chen ,  Yu Zhang ,  Yuxuan Wang ,  Russell John Wyatt Skerry-Ryan ,  Ryan M. Rifkin ,  Ioannis Agiomyrgiannakis

IPC: G10L25/30 ,  G10L13/047 ,  G10L13/08 ,  G06N7/00 ,  G06N3/08 ,  G06N3/04 ,  G06N5/04 ,  G10L25/18

Abstract: Methods, systems, and computer program products for generating, from an input character sequence, an output sequence of audio data representing the input character sequence. The output sequence of audio data includes a respective audio output sample for each of a number of time steps. One example method includes, for each of the time steps: generating a mel-frequency spectrogram for the time step by processing a representation of a respective portion of the input character sequence using a decoder neural network; generating a probability distribution over a plurality of possible audio output samples for the time step by processing the mel-frequency spectrogram for the time step using a vocoder neural network; and selecting the audio output sample for the time step from the possible audio output samples in accordance with the probability distribution.

公开(公告)号：US20250095634A1

公开(公告)日：2025-03-20

申请号：US18965193

申请日：2024-12-02

Applicant: Google LLC

Inventor： Bo Li ,  Tara N. Sainath ,  Ruoming Pang ,  Shuo-yiin Chang ,  Qiumin Xu ,  Trevor Strohman ,  Vince Chen ,  Qiao Liang ,  Heguang Liu ,  Yanzhang He ,  Parisa Haghani ,  Sameer Bidichandani

IPC: G10L15/00 ,  G10L15/06 ,  G10L15/22 ,  G10L15/30

Abstract: A method includes receiving a sequence of acoustic frames characterizing one or more utterances as input to a multilingual automated speech recognition (ASR) model. The method also includes generating a higher order feature representation for a corresponding acoustic frame. The method also includes generating a hidden representation based on a sequence of non-blank symbols output by a final softmax layer. The method also includes generating a probability distribution over possible speech recognition hypotheses based on the hidden representation generated by the prediction network at each of the plurality of output steps and the higher order feature representation generated by the encoder at each of the plurality of output steps. The method also includes predicting an end of utterance (EOU) token at an end of each utterance. The method also includes classifying each acoustic frame as either speech, initial silence, intermediate silence, or final silence.

公开(公告)号：US20240362453A1

公开(公告)日：2024-10-31

申请号：US18766038

申请日：2024-07-08

Applicant: Google LLC

Inventor： Anmol Gulati ,  Weikeng Qin ,  Zhengdong Zhang ,  Ruoming Pang ,  Niki Parmar ,  Jiahui Yu ,  Wei Han ,  Chung-Cheng Chiu ,  Yu Zhang ,  Yonghui Wu ,  Shibo Wang

IPC: G06N3/04 ,  G06N20/00 ,  G10L15/16

CPC classification number: G06N3/04 ,  G06N20/00 ,  G10L15/16

Abstract: Systems and methods can utilize a conformer model to process a data set for various data processing tasks, including, but not limited to, speech recognition, sound separation, protein synthesis determination, video or other image set analysis, and natural language processing. The conformer model can use feed-forward blocks, a self-attention block, and a convolution block to process data to learn global interactions and relative-offset-based local correlations of the input data.

公开(公告)号：US12079703B2

公开(公告)日：2024-09-03

申请号：US17139525

申请日：2020-12-31

Applicant: Google LLC

Inventor： Anmol Gulati ,  Ruoming Pang ,  Niki Parmar ,  Jiahui Yu ,  Wei Han ,  Chung-Cheng Chiu ,  Yu Zhang ,  Yonghui Wu ,  Shibo Wang ,  Weikeng Qin ,  Zhengdong Zhang

IPC: G06N3/04 ,  G06N20/00 ,  G10L15/16

CPC classification number: G06N3/04 ,  G06N20/00 ,  G10L15/16

Abstract: Systems and methods can utilize a conformer model to process a data set for various data processing tasks, including, but not limited to, speech recognition, sound separation, protein synthesis determination, video or other image set analysis, and natural language processing. The conformer model can use feed-forward blocks, a self-attention block, and a convolution block to process data to learn global interactions and relative-offset-based local correlations of the input data.

公开(公告)号：US12073824B2

公开(公告)日：2024-08-27

申请号：US17616135

申请日：2020-12-03

Applicant: GOOGLE LLC

Inventor： Tara N. Sainath ,  Yanzhang He ,  Bo Li ,  Arun Narayanan ,  Ruoming Pang ,  Antoine Jean Bruguier ,  Shuo-Yiin Chang ,  Wei Li

IPC: G10L15/00 ,  G06N3/08 ,  G10L15/05 ,  G10L15/06 ,  G10L15/16 ,  G10L15/22

CPC classification number: G10L15/16 ,  G06N3/08 ,  G10L15/05 ,  G10L15/063 ,  G10L15/22 ,  G10L2015/0635

Abstract: Two-pass automatic speech recognition (ASR) models can be used to perform streaming on-device ASR to generate a text representation of an utterance captured in audio data. Various implementations include a first-pass portion of the ASR model used to generate streaming candidate recognition(s) of an utterance captured in audio data. For example, the first-pass portion can include a recurrent neural network transformer (RNN-T) decoder. Various implementations include a second-pass portion of the ASR model used to revise the streaming candidate recognition(s) of the utterance and generate a text representation of the utterance. For example, the second-pass portion can include a listen attend spell (LAS) decoder. Various implementations include a shared encoder shared between the RNN-T decoder and the LAS decoder.

公开(公告)号：US20240273336A1

公开(公告)日：2024-08-15

申请号：US18430483

申请日：2024-02-01

Applicant: Google LLC

Inventor： Mingxing Tan ,  Quoc Le ,  Bo Chen ,  Vijay Vasudevan ,  Ruoming Pang

IPC: G06N3/04 ,  G06F17/15 ,  G06N3/044 ,  G06N3/045 ,  G06N3/084 ,  G06N20/10

CPC classification number: G06N3/04 ,  G06F17/15 ,  G06N3/044 ,  G06N3/084 ,  G06N20/10 ,  G06N3/045

Abstract: The present disclosure is directed to an automated neural architecture search approach for designing new neural network architectures such as, for example, resource-constrained mobile CNN models. In particular, the present disclosure provides systems and methods to perform neural architecture search using a novel factorized hierarchical search space that permits layer diversity throughout the network, thereby striking the right balance between flexibility and search space size. The resulting neural architectures are able to be run relatively faster and using relatively fewer computing resources (e.g., less processing power, less memory usage, less power consumption, etc.), all while remaining competitive with or even exceeding the performance (e.g., accuracy) of current state-of-the-art mobile-optimized models.

公开(公告)号：US12027154B2

公开(公告)日：2024-07-02

申请号：US18167050

申请日：2023-02-09

Applicant: Google LLC

Inventor： Tara N. Sainath ,  Basilio Garcia Castillo ,  David Rybach ,  Trevor Strohman ,  Ruoming Pang

IPC: G10L25/30 ,  G10L15/06 ,  G10L25/78

CPC classification number: G10L15/063 ,  G10L25/30 ,  G10L25/78

Abstract: A method includes receiving a training example that includes audio data representing a spoken utterance and a ground truth transcription. For each word in the spoken utterance, the method also includes inserting a placeholder symbol before the respective word identifying a respective ground truth alignment for a beginning and an end of the respective word, determining a beginning word piece and an ending word piece, and generating a first constrained alignment for the beginning word piece and a second constrained alignment for the ending word piece. The first constrained alignment is aligned with the ground truth alignment for the beginning of the respective word and the second constrained alignment is aligned with the ground truth alignment for the ending of the respective word. The method also includes constraining an attention head of a second pass decoder by applying the first and second constrained alignments.

公开(公告)号：US20240112667A1

公开(公告)日：2024-04-04

申请号：US18525475

申请日：2023-11-30

Applicant: Google LLC

Inventor： Ye Jia ,  Zhifeng Chen ,  Yonghui Wu ,  Jonathan Shen ,  Ruoming Pang ,  Ron J. Weiss ,  Ignacio Lopez Moreno ,  Fei Ren ,  Yu Zhang ,  Quan Wang ,  Patrick An Phu Nguyen

IPC: G10L13/04 ,  G10L17/04 ,  G10L19/00

CPC classification number: G10L13/04 ,  G10L17/04 ,  G10L19/00 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for speech synthesis. The methods, systems, and apparatus include actions of obtaining an audio representation of speech of a target speaker, obtaining input text for which speech is to be synthesized in a voice of the target speaker, generating a speaker vector by providing the audio representation to a speaker encoder engine that is trained to distinguish speakers from one another, generating an audio representation of the input text spoken in the voice of the target speaker by providing the input text and the speaker vector to a spectrogram generation engine that is trained using voices of reference speakers to generate audio representations, and providing the audio representation of the input text spoken in the voice of the target speaker for output.