公开(公告)号：US20240127791A1

公开(公告)日：2024-04-18

申请号：US18516069

申请日：2023-11-21

申请人： Google LLC

发明人： Samuel Bengio ,  Yuxuan Wang ,  Zongheng Yang ,  Zhifeng Chen ,  Yonghui Wu ,  Ioannis Agiomyrgiannakis ,  Ron J. Weiss ,  Navdeep Jaitly ,  Ryan M. Rifkin ,  Robert Andrew James Clark ,  Quoc V. Le ,  Russell J. Ryan ,  Ying Xiao

IPC分类号： G10L13/08 ,  G06N3/045 ,  G06N3/08 ,  G06N3/084 ,  G10L13/04 ,  G10L15/16 ,  G10L25/18 ,  G10L25/30

CPC分类号： G10L13/08 ,  G06N3/045 ,  G06N3/08 ,  G06N3/084 ,  G10L13/04 ,  G10L15/16 ,  G10L25/18 ,  G10L25/30

摘要： Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating speech from text. One of the systems includes one or more computers and one or more storage devices storing instructions that when executed by one or more computers cause the one or more computers to implement: a sequence-to-sequence recurrent neural network configured to: receive a sequence of characters in a particular natural language, and process the sequence of characters to generate a spectrogram of a verbal utterance of the sequence of characters in the particular natural language; and a subsystem configured to: receive the sequence of characters in the particular natural language, and provide the sequence of characters as input to the sequence-to-sequence recurrent neural network to obtain as output the spectrogram of the verbal utterance of the sequence of characters in the particular natural language.

公开(公告)号：US20230237995A1

公开(公告)日：2023-07-27

申请号：US18194586

申请日：2023-03-31

申请人： Google LLC

发明人： Rohit Prakash Prabhavalkar ,  Tara N. Sainath ,  Younghui Wu ,  Patrick An Phu Nguyen ,  Zhifeng Chen ,  Chung-Cheng Chiu ,  Anjuli Kannan

IPC分类号： G10L15/197 ,  G10L15/16 ,  G10L15/06 ,  G10L15/02 ,  G10L15/22

CPC分类号： G10L15/197 ,  G10L15/16 ,  G10L15/063 ,  G10L15/02 ,  G10L15/22 ,  G10L2015/025

摘要： Methods, systems, and apparatus, including computer programs encoded on computer-readable storage media, for speech recognition using attention-based sequence-to-sequence models. In some implementations, audio data indicating acoustic characteristics of an utterance is received. A sequence of feature vectors indicative of the acoustic characteristics of the utterance is generated. The sequence of feature vectors is processed using a speech recognition model that has been trained using a loss function that uses a set of speech recognition hypothesis samples, the speech recognition model including an encoder, an attention module, and a decoder. The encoder and decoder each include one or more recurrent neural network layers. A sequence of output vectors representing distributions over a predetermined set of linguistic units is obtained. A transcription for the utterance is obtained based on the sequence of output vectors. Data indicating the transcription of the utterance is provided.

公开(公告)号：US11508147B2

公开(公告)日：2022-11-22

申请号：US16812154

申请日：2020-03-06

申请人： Google LLC

发明人： Jonathon Shlens ,  Vijay Vasudevan ,  Jiquan Ngiam ,  Wei Han ,  Zhifeng Chen ,  Brandon Chauloon Yang ,  Benjamin James Caine ,  Zhengdong Zhang ,  Christoph Sprunk ,  Ouais Alsharif ,  Junhua Mao ,  Chen Wu

IPC分类号： G06V20/10 ,  G06T17/00 ,  G01S17/89 ,  G06N3/04

摘要： Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for processing data generated by a sensing system that rotationally senses an environment. In one aspect, a method comprises partitioning a predetermined period of time into a plurality of sub-periods, wherein the predetermined period of time is a period of time for which data generated by the sensing system constitutes a complete rotational sensing of the environment; for each sub-period: receiving current data generated by the sensing system during the sub-period and characterizing a respective partial scene of the environment; processing the current data using an object detection neural network to generate a current object detection output that is specific to the respective partial scene of the environment.

公开(公告)号：US20220180193A1

公开(公告)日：2022-06-09

申请号：US17547143

申请日：2021-12-09

申请人： Google LLC

发明人： Benjamin James Caine ,  Rebecca Dawn Roelofs ,  Jonathon Shlens ,  Zhifeng Chen ,  Jiquan Ngiam ,  Vijay Vasudevan

IPC分类号： G06N3/08

摘要： Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network to perform 3D object detection. One of the methods includes training a student neural network to perform 3D object detection using pseudo-labels generated by a teacher neural network.

公开(公告)号：US20220083746A1

公开(公告)日：2022-03-17

申请号：US17459041

申请日：2021-08-27

申请人： Google LLC

发明人： Zhifeng Chen ,  Macduff Richard Hughes ,  Yonghui Wu ,  Michael Schuster ,  Xu Chen ,  Llion Owen Jones ,  Niki J. Parmar ,  George Foster ,  Orhan Firat ,  Ankur Bapna ,  Wolfgang Macherey ,  Melvin Jose Johnson Premkumar

IPC分类号： G06F40/58 ,  G06N3/08

摘要： Methods, systems, and apparatus, including computer programs encoded on computer storage media, for machine translation using neural networks. In some implementations, a text in one language is translated into a second language using a neural network model. The model can include an encoder neural network comprising a plurality of bidirectional recurrent neural network layers. The encoding vectors are processed using a multi-headed attention module configured to generate multiple attention context vectors for each encoding vector. A decoder neural network generates a sequence of decoder output vectors using the attention context vectors. The decoder output vectors can represent distributions over various language elements of the second language, allowing a translation of the text into the second language to be determined based on the sequence of decoder output vectors.

公开(公告)号：US20210295858A1

公开(公告)日：2021-09-23

申请号：US17222736

申请日：2021-04-05

申请人： Google LLC

发明人： Yonghui Wu ,  Jonathan Shen ,  Ruoming Pang ,  Ron J. Weiss ,  Michael Schuster ,  Navdeep Jaitly ,  Zongheng Yang ,  Zhifeng Chen ,  Yu Zhang ,  Yuxuan Wang ,  Russell John Wyatt Skerry-Ryan ,  Ryan M. Rifkin ,  Ioannis Agiomyrgiannakis

IPC分类号： G10L25/30 ,  G10L13/047 ,  G10L13/08 ,  G06N7/00 ,  G06N3/08 ,  G06N3/04 ,  G06N5/04 ,  G10L25/18

摘要： Methods, systems, and computer program products for generating, from an input character sequence, an output sequence of audio data representing the input character sequence. The output sequence of audio data includes a respective audio output sample for each of a number of time steps. One example method includes, for each of the time steps: generating a mel-frequency spectrogram for the time step by processing a representation of a respective portion of the input character sequence using a decoder neural network; generating a probability distribution over a plurality of possible audio output samples for the time step by processing the mel-frequency spectrogram for the time step using a vocoder neural network; and selecting the audio output sample for the time step from the possible audio output samples in accordance with the probability distribution.

公开(公告)号：US11107457B2

公开(公告)日：2021-08-31

申请号：US16696101

申请日：2019-11-26

申请人： Google LLC

发明人： Samuel Bengio ,  Yuxuan Wang ,  Zongheng Yang ,  Zhifeng Chen ,  Yonghui Wu ,  Ioannis Agiomyrgiannakis ,  Ron J. Weiss ,  Navdeep Jaitly ,  Ryan M. Rifkin ,  Robert Andrew James Clark ,  Quoc V. Le ,  Russell J. Ryan ,  Ying Xiao

IPC分类号： G10L13/06 ,  G10L13/08 ,  G06N3/08 ,  G10L25/18 ,  G10L25/30 ,  G10L13/04 ,  G06N3/04 ,  G10L15/16

摘要： Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating speech from text. One of the systems includes one or more computers and one or more storage devices storing instructions that when executed by one or more computers cause the one or more computers to implement: a sequence-to-sequence recurrent neural network configured to: receive a sequence of characters in a particular natural language, and process the sequence of characters to generate a spectrogram of a verbal utterance of the sequence of characters in the particular natural language; and a subsystem configured to: receive the sequence of characters in the particular natural language, and provide the sequence of characters as input to the sequence-to-sequence recurrent neural network to obtain as output the spectrogram of the verbal utterance of the sequence of characters in the particular natural language.

公开(公告)号：US20210209315A1

公开(公告)日：2021-07-08

申请号：US17056554

申请日：2020-03-07

申请人： Google LLC

发明人： Ye Jia ,  Zhifeng Chen ,  Yonghui Wu ,  Melvin Johnson ,  Fadi Biadsy ,  Ron Weiss ,  Wolfgang Macherey

IPC分类号： G06F40/47 ,  G06F40/58

摘要： The present disclosure provides systems and methods that train and use machine-learned models such as, for example, sequence-to-sequence models, to perform direct and text-free speech-to-speech translation. In particular, aspects of the present disclosure provide an attention-based sequence-to-sequence neural network which can directly translate speech from one language into speech in another language, without relying on an intermediate text representation.

公开(公告)号：US20210042620A1

公开(公告)日：2021-02-11

申请号：US16989787

申请日：2020-08-10

申请人： Google LLC

发明人： Zhifeng Chen ,  Yanping Huang ,  Youlong Cheng ,  HyoukJoong Lee ,  Dehao Chen ,  Jiquan Ngiam

IPC分类号： G06N3/08 ,  G06N3/04

摘要： Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training giant neural networks. One of the methods includes obtaining data specifying a partitioning of the neural network into N composite layers that form a sequence of composite layers, wherein each composite layer comprises a distinct plurality of layers from the multiple network layers of the neural network; obtaining data assigning each of the N composite layers to one or more computing devices from a set of N computing devices; partitioning a mini-batch of training examples into a plurality of micro-batches; and training the neural network, comprising: performing a forward pass through the neural network until output activations have been computed for each micro-batch for a final composite layer in the sequence, and performing a backward pass through the neural network until output gradients have been computed for each micro-batch for the first composite layer in the sequence.

公开(公告)号：US20200380952A1

公开(公告)日：2020-12-03

申请号：US16855042

申请日：2020-04-22

申请人： Google LLC

发明人： Yu Zhang ,  Ron J. Weiss ,  Byungha Chun ,  Yonghui Wu ,  Zhifeng Chen ,  Russell John Wyatt Skerry-Ryan ,  Ye Jia ,  Andrew M. Rosenberg ,  Bhuvana Ramabhadran

IPC分类号： G10L13/047

摘要： A method includes receiving an input text sequence to be synthesized into speech in a first language and obtaining a speaker embedding, the speaker embedding specifying specific voice characteristics of a target speaker for synthesizing the input text sequence into speech that clones a voice of the target speaker. The target speaker includes a native speaker of a second language different than the first language. The method also includes generating, using a text-to-speech (TTS) model, an output audio feature representation of the input text by processing the input text sequence and the speaker embedding. The output audio feature representation includes the voice characteristics of the target speaker specified by the speaker embedding.