-
公开(公告)号:US20210249027A1
公开(公告)日:2021-08-12
申请号:US17170657
申请日:2021-02-08
Applicant: Google LLC
Inventor: Neil Zeghidour , David Grangier
IPC: G10L21/028 , G10L17/04 , G10L17/18 , G10L21/0316 , G06N3/04 , G06N3/08
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing speech separation. One of the methods includes obtaining a recording comprising speech from a plurality of speakers; processing the recording using a speaker neural network having speaker parameter values and configured to process the recording in accordance with the speaker parameter values to generate a plurality of per-recording speaker representations, each speaker representation representing features of a respective identified speaker in the recording; and processing the per-recording speaker representations and the recording using a separation neural network having separation parameter values and configured to process the recording and the speaker representations in accordance with the separation parameter values to generate, for each speaker representation, a respective predicted isolated audio signal that corresponds to speech of one of the speakers in the recording.
-
公开(公告)号:US20250157456A1
公开(公告)日:2025-05-15
申请号:US18832325
申请日:2024-01-26
Applicant: Google LLC
Inventor: Evgeny Kharitonov , Damien Vincent , Zalán Borsos , Raphaël Marinier , Olivier Claude Pietquin , Matthew Sharifi , Marco Tagliasacchi , Neil Zeghidour
IPC: G10L13/027 , G06F40/284 , G06F40/30 , G10L13/08
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating an audio signal from input text. In one aspect, a method comprises receiving a request to convert input text into an audio signal, wherein the input text comprises multiple tokenized text inputs, generating, using a first generative neural network, a semantic representation of the tokenized text inputs comprising semantic tokens representing semantic content of the tokenized text inputs, each semantic token being selected from a vocabulary of semantic tokens, generating, using a second generative neural network and conditioned on at least the semantic representation, an acoustic representation of the semantic representation comprising one or more respective acoustic tokens representing acoustic properties of the audio signal, and processing the acoustic representation using a decoder neural network to generate the audio signal.
-
公开(公告)号:US20250022477A1
公开(公告)日:2025-01-16
申请号:US18278746
申请日:2023-03-16
Applicant: Google LLC
Inventor: Ahmed Omran , Neil Zeghidour , Zalán Borsos , Félix de Chaumont Quitry , Marco Tagliasacchi
IPC: G10L19/038 , G10L25/30 , G10L25/60
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an encoder neural network and a decoder neural network. In one aspect, a method includes obtaining a first initial audio waveform and a first noisy audio waveform, obtaining a second initial audio waveform and a second noisy audio waveform, processing the first noisy audio waveform and the second noisy audio waveform using an encoder neural network, generating a blended embedding by concatenating: (i) clean feature dimensions from an embedding of the first noisy audio waveform, and (ii) noise feature dimensions from an embedding of the second noisy audio waveform, processing the blended embedding using a decoder neural network to generate a reconstructed audio waveform, determining gradients of an objective function; and updating parameter values of the encoder neural network and the decoder neural network using the gradients.
-
公开(公告)号:US12198710B2
公开(公告)日:2025-01-14
申请号:US18400992
申请日:2023-12-29
Applicant: Google LLC
Inventor: Neil Zeghidour , Marco Tagliasacchi , Dominik Roblek
IPC: G10L19/038 , G06N3/045 , G06N3/08 , G10L19/00 , G10L25/30
Abstract: Methods, systems and apparatus, including computer programs encoded on computer storage media. According to one aspect, there is provided a method comprising: receiving a new input; processing the new input using an encoder neural network to generate a feature vector representing the new input; and generating a coded representation of the feature vector using a sequence of vector quantizers that are each associated with a respective codebook of code vectors, wherein the coded representation of the feature vector identifies a plurality of code vectors, including a respective code vector from the codebook of each vector quantizer, that define a quantized representation of the feature vector.
-
公开(公告)号:US20240428056A1
公开(公告)日:2024-12-26
申请号:US18750973
申请日:2024-06-21
Applicant: Google LLC
Inventor: Paul Kishan Rubenstein , Matthew Sharifi , Alexandru Tudor , Chulayuth Asawaroengchai , Duc Dung Nguyen , Marco Tagliasacchi , Neil Zeghidour , Zalán Borsos , Christian Frank , Dalia Salem Hassan Fahmy Elbadawy , Hannah Raphaelle Muckenhirn , Dirk Ryan Padfield , Damien Vincent , Evgeny Kharitonov , Michelle Dana Tadmor , Mihajlo Velimirovic , Feifan Chen , Victoria Zayats
IPC: G06N3/0475 , G10L25/30
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing tasks. One of the methods includes obtaining a sequence of input tokens, where each token is selected from a vocabulary of tokens that includes text tokens and audio tokens, and wherein the sequence of input tokens includes tokens that describe a task to be performed and data for performing the task; generating a sequence of embeddings by embedding each token in the sequence of input tokens in an embedding space; and processing the sequence of embeddings using a language model neural network to generate a sequence of output tokens for the task, where each token is selected from the vocabulary.
-
公开(公告)号:US20240144957A1
公开(公告)日:2024-05-02
申请号:US18544647
申请日:2023-12-19
Applicant: Google LLC
Inventor: David Grangier , Neil Zeghidour , Oliver Teboul
CPC classification number: G10L25/78 , G06N3/04 , G10L15/063 , G10L15/07 , G10L17/18 , G10L19/008
Abstract: A method includes receiving an input audio signal corresponding to utterances spoken by multiple speakers. The method also includes encoding the input audio signal into a sequence of T temporal embeddings. During each of a plurality of iterations each corresponding to a respective speaker of the multiple speakers, the method includes selecting a respective speaker embedding for the respective speaker by determining a probability that the corresponding temporal embedding includes a presence of voice activity by a single new speaker for which a speaker embedding was not previously selected during a previous iteration and selecting the respective speaker embedding for the respective speaker as the temporal embedding. The method also includes, at each time step, predicting a respective voice activity indicator for each respective speaker of the multiple speakers based on the respective speaker embeddings selected during the plurality of iterations and the temporal embedding.
-
公开(公告)号:US20240078412A1
公开(公告)日:2024-03-07
申请号:US18463092
申请日:2023-09-07
Applicant: Google LLC
Inventor: Neil Zeghidour , David Grangier , Marco Tagliasacchi , Raphaël Marinier , Olivier Teboul , Zalán Borsos
IPC: G06N3/0455 , G06N3/0475
CPC classification number: G06N3/0455 , G06N3/0475
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a prediction of an audio signal. One of the methods includes receiving a request to generate an audio signal; obtaining a semantic representation of the audio signal; generating, using one or more generative neural networks and conditioned on at least the semantic representation, an acoustic representation of the audio signal; and processing at least the acoustic representation using a decoder neural network to generate the prediction of the audio signal.
-
公开(公告)号:US11915689B1
公开(公告)日:2024-02-27
申请号:US18463196
申请日:2023-09-07
Applicant: Google LLC
Inventor: Andrea Agostinelli , Timo Immanuel Denk , Antoine Caillon , Neil Zeghidour , Jesse Engel , Mauro Verzetti , Christian Frank , Zalán Borsos , Matthew Sharifi , Adam Joseph Roberts , Marco Tagliasacchi
CPC classification number: G10L15/16 , G10H1/0008 , G10L15/063 , G10L15/1815 , G10H2210/056 , G10H2250/311
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a prediction of an audio signal. One of the methods includes receiving a request to generate an audio signal conditioned on an input; processing the input using an embedding neural network to map the input to one or more embedding tokens; generating a semantic representation of the audio signal; generating, using one or more generative neural networks and conditioned on at least the semantic representation and the embedding tokens, an acoustic representation of the audio signal; and processing at least the acoustic representation using a decoder neural network to generate the prediction of the audio signal.
-
公开(公告)号:US20230379645A1
公开(公告)日:2023-11-23
申请号:US17748356
申请日:2022-05-19
Applicant: Google LLC
Inventor: Rajeev Conrad Nongpiur , Qian Zhang , Andrew James Sutter , Kung-Wei Liu , Jihan Li , Hélène Bahu , Leonardo Kusumo , Sze Chie Lim , Marco Tagliasacchi , Neil Zeghidour , Michael Takezo Chinen
CPC classification number: H04S7/30 , G10L19/008 , H04R5/027 , H04R3/005 , H04S3/008 , G06N20/00 , H04S2420/11 , H04S2400/11 , H04S2400/15 , H04S2420/03 , H04S2400/01 , H04R2420/07
Abstract: The technology generally relates to spatial audio communication between devices. For example, a first device and a second device may be connected via a communication link. The first device may capture audio signals in an environment through two or more microphones. The first device may encode the captured audio with spatial configuration data. The first device may transmit the encoded audio via the communication link to the second device. The second device may decode the encoded audio into binaural or ambisonic audio to be output by one or more speakers of the second device. The binaural or ambisonic audio may be converted into spatial audio to be output. The second device may output the binaural or spatial audio to create an immersive listening experience.
-
公开(公告)号:US20230186927A1
公开(公告)日:2023-06-15
申请号:US18106094
申请日:2023-02-06
Applicant: Google LLC
Inventor: Neil Zeghidour , Marco Tagliasacchi , Dominik Roblek
IPC: G10L19/038 , G06N3/045 , G06N3/08 , G10L25/30
CPC classification number: G10L19/038 , G06N3/045 , G06N3/08 , G10L25/30 , G10L2019/0002
Abstract: Methods, systems and apparatus, including computer programs encoded on computer storage media. One of the methods includes receiving an audio waveform that includes a respective audio sample for each of a plurality of time steps, processing the audio waveform using an encoder neural network to generate a plurality of feature vectors representing the audio waveform, generating a respective coded representation of each of the plurality of feature vectors using a plurality of vector quantizers that are each associated with a respective codebook of code vectors, wherein the respective coded representation of each feature vector identifies a plurality of code vectors, including a respective code vector from the codebook of each vector quantizer, that define a quantized representation of the feature vector, and generating a compressed representation of the audio waveform by compressing the respective coded representation of each of the plurality of feature vectors.
-
-
-
-
-
-
-
-
-