公开(公告)号：US20240395239A1

公开(公告)日：2024-11-28

申请号：US18795734

申请日：2024-08-06

Applicant: Google LLC

Inventor： Daisy Antonia Stanton ,  Sean Matthew Shannon ,  Soroosh Mariooryad ,  Russell John Wyatt Skerry-Ryan ,  Eric Dean Battenberg ,  Thomas Edward Bagby ,  David Teh-Hwa Kao

IPC: G10L13/08 ,  G06N3/08 ,  G10L13/027

Abstract: Systems and methods for text-to-speech with novel speakers can obtain text data and output audio data. The input text data may be input along with one or more speaker preferences. The speaker preferences can include speaker characteristics. The speaker preferences can be processed by a machine-learned model conditioned on a learned prior distribution to determine a speaker embedding. The speaker embedding can then be processed with the text data to generate an output that includes audio data descriptive of the text data spoken by a novel speaker.

公开(公告)号：US12067969B2

公开(公告)日：2024-08-20

申请号：US18302764

申请日：2023-04-18

Applicant: Google LLC

Inventor： Eric Dean Battenberg ,  Daisy Stanton ,  Russell John Wyatt Skerry-Ryan ,  Soroosh Mariooryad ,  David Teh-Hwa Kao ,  Thomas Edward Bagby ,  Sean Matthew Shannon

IPC: G10L13/00 ,  G10L13/047 ,  G10L13/10

CPC classification number: G10L13/047 ,  G10L13/10

Abstract: A method for estimating an embedding capacity includes receiving, at a deterministic reference encoder, a reference audio signal, and determining a reference embedding corresponding to the reference audio signal, the reference embedding having a corresponding embedding dimensionality. The method also includes measuring a first reconstruction loss as a function of the corresponding embedding dimensionality of the reference embedding and obtaining a variational embedding from a variational posterior. The variational embedding has a corresponding embedding dimensionality and a specified capacity. The method also includes measuring a second reconstruction loss as a function of the corresponding embedding dimensionality of the variational embedding and estimating a capacity of the reference embedding by comparing the first measured reconstruction loss for the reference embedding relative to the second measured reconstruction loss for the variational embedding having the specified capacity.

公开(公告)号：US20230206898A1

公开(公告)日：2023-06-29

申请号：US17673417

申请日：2022-02-16

Applicant: Google LLC

Inventor： Daisy Antonia Stanton ,  Sean Matthew Shannon ,  Soroosh Mariooryad ,  Russell John-Wyatt Skerry-Ryan ,  Eric Dean Battenberg ,  Thomas Edward Bagby ,  David Teh-Hwa Kao

IPC: G10L13/08 ,  G06N3/08 ,  G10L13/027

CPC classification number: G10L13/086 ,  G06N3/08 ,  G10L13/027

Abstract: Systems and methods for text-to-speech with novel speakers can obtain text data and output audio data. The input text data may be input along with one or more speaker preferences. The speaker preferences can include speaker characteristics. The speaker preferences can be processed by a machine-learned model conditioned on a learned prior distribution to determine a speaker embedding. The speaker embedding can then be processed with the text data to generate an output that includes audio data descriptive of the text data spoken by a novel speaker.

公开(公告)号：US11222621B2

公开(公告)日：2022-01-11

申请号：US16879714

申请日：2020-05-20

Applicant: Google LLC

Inventor： Eric Dean Battenberg ,  Daisy Stanton ,  Russell John Wyatt Skerry-Ryan ,  Soroosh Mariooryad ,  David Teh-hwa Kao ,  Thomas Edward Bagby ,  Sean Matthew Shannon

IPC: G10L15/22 ,  G06N3/00 ,  G06N3/08 ,  G10L13/047 ,  G10L13/10

Abstract: A method for estimating an embedding capacity includes receiving, at a deterministic reference encoder, a reference audio signal, and determining a reference embedding corresponding to the reference audio signal, the reference embedding having a corresponding embedding dimensionality. The method also includes measuring a first reconstruction loss as a function of the corresponding embedding dimensionality of the reference embedding and obtaining a variational embedding from a variational posterior. The variational embedding has a corresponding embedding dimensionality and a specified capacity. The method also includes measuring a second reconstruction loss as a function of the corresponding embedding dimensionality of the variational embedding and estimating a capacity of the reference embedding by comparing the first measured reconstruction loss for the reference embedding relative to the second measured reconstruction loss for the variational embedding having the specified capacity.

公开(公告)号：US20240395238A1

公开(公告)日：2024-11-28

申请号：US18796738

申请日：2024-08-07

Applicant: Google LLC

Inventor： Eric Dean Battenberg ,  Daisy Stanton ,  Russell John Wyatt Skerry-Ryan ,  Soroosh Mariooryad ,  David Teh-hwa Kao ,  Thomas Edward Bagby ,  Sean Matthew Shannon

IPC: G10L13/047 ,  G10L13/10

Abstract: A method for estimating an embedding capacity includes receiving, at a deterministic reference encoder, a reference audio signal, and determining a reference embedding corresponding to the reference audio signal, the reference embedding having a corresponding embedding dimensionality. The method also includes measuring a first reconstruction loss as a function of the corresponding embedding dimensionality of the reference embedding and obtaining a variational embedding from a variational posterior. The variational embedding has a corresponding embedding dimensionality and a specified capacity. The method also includes measuring a second reconstruction loss as a function of the corresponding embedding dimensionality of the variational embedding and estimating a capacity of the reference embedding by comparing the first measured reconstruction loss for the reference embedding relative to the second measured reconstruction loss for the variational embedding having the specified capacity.

公开(公告)号：US12087275B2

公开(公告)日：2024-09-10

申请号：US17673417

申请日：2022-02-16

Applicant: Google LLC

Inventor： Daisy Antonia Stanton ,  Sean Matthew Shannon ,  Soroosh Mariooryad ,  Russell John-Wyatt Skerry-Ryan ,  Eric Dean Battenberg ,  Thomas Edward Bagby ,  David Teh-Hwa Kao

IPC: G10L13/08 ,  G06N3/08 ,  G10L13/027

CPC classification number: G10L13/086 ,  G06N3/08 ,  G10L13/027

Abstract: Systems and methods for text-to-speech with novel speakers can obtain text data and output audio data. The input text data may be input along with one or more speaker preferences. The speaker preferences can include speaker characteristics. The speaker preferences can be processed by a machine-learned model conditioned on a learned prior distribution to determine a speaker embedding. The speaker embedding can then be processed with the text data to generate an output that includes audio data descriptive of the text data spoken by a novel speaker.

公开(公告)号：US11646010B2

公开(公告)日：2023-05-09

申请号：US17643455

申请日：2021-12-09

Applicant: Google LLC

Inventor： Eric Dean Battenberg ,  Daisy Stanton ,  Russell John Wyatt Skerry-Ryan ,  Soroosh Mariooryad ,  David Teh-Hwa Kao ,  Thomas Edward Bagby ,  Sean Matthew Shannon

IPC: G10L25/63 ,  G06F40/30 ,  G10L13/047 ,  G10L13/10

CPC classification number: G10L13/047 ,  G10L13/10

Abstract: A method for estimating an embedding capacity includes receiving, at a deterministic reference encoder, a reference audio signal, and determining a reference embedding corresponding to the reference audio signal, the reference embedding having a corresponding embedding dimensionality. The method also includes measuring a first reconstruction loss as a function of the corresponding embedding dimensionality of the reference embedding and obtaining a variational embedding from a variational posterior. The variational embedding has a corresponding embedding dimensionality and a specified capacity. The method also includes measuring a second reconstruction loss as a function of the corresponding embedding dimensionality of the variational embedding and estimating a capacity of the reference embedding by comparing the first measured reconstruction loss for the reference embedding relative to the second measured reconstruction loss for the variational embedding having the specified capacity.

公开(公告)号：US20220101826A1

公开(公告)日：2022-03-31

申请号：US17643455

申请日：2021-12-09

Applicant: Google LLC

Inventor： Eric Dean Battenberg ,  Daisy Stanton ,  Russell John Wyatt Skerry-Ryan ,  Soroosh Mariooryad ,  David Teh-Hwa Kao ,  Thomas Edward Bagby ,  Sean Matthew Shannon

IPC: G10L13/047 ,  G10L13/10

Abstract: A method for estimating an embedding capacity includes receiving, at a deterministic reference encoder, a reference audio signal, and determining a reference embedding corresponding to the reference audio signal, the reference embedding having a corresponding embedding dimensionality. The method also includes measuring a first reconstruction loss as a function of the corresponding embedding dimensionality of the reference embedding and obtaining a variational embedding from a variational posterior. The variational embedding has a corresponding embedding dimensionality and a specified capacity. The method also includes measuring a second reconstruction loss as a function of the corresponding embedding dimensionality of the variational embedding and estimating a capacity of the reference embedding by comparing the first measured reconstruction loss for the reference embedding relative to the second measured reconstruction loss for the variational embedding having the specified capacity.

公开(公告)号：US20200372897A1

公开(公告)日：2020-11-26

申请号：US16879714

申请日：2020-05-20

Applicant: Google LLC

Inventor： Eric Dean Battenberg ,  Daisy Stanton ,  Russell John Wyatt Skerry-Ryan ,  Soroosh Mariooryad ,  David Teh-hwa Kao ,  Thomas Edward Bagby ,  Sean Matthew Shannon

IPC: G10L13/047 ,  G10L13/10

Abstract: A method for estimating an embedding capacity includes receiving, at a deterministic reference encoder, a reference audio signal, and determining a reference embedding corresponding to the reference audio signal, the reference embedding having a corresponding embedding dimensionality. The method also includes measuring a first reconstruction loss as a function of the corresponding embedding dimensionality of the reference embedding and obtaining a variational embedding from a variational posterior. The variational embedding has a corresponding embedding dimensionality and a specified capacity. The method also includes measuring a second reconstruction loss as a function of the corresponding embedding dimensionality of the variational embedding and estimating a capacity of the reference embedding by comparing the first measured reconstruction loss for the reference embedding relative to the second measured reconstruction loss for the variational embedding having the specified capacity.

公开(公告)号：US20250131273A1

公开(公告)日：2025-04-24

申请号：US18696052

申请日：2023-09-27

Applicant: Google LLC

Inventor： Soroosh Mariooryad ,  Sean Matthew Shannon ,  Thomas Edward Bagby ,  Siyuan Ma ,  David Teh-Hwa Kao ,  Daisy Antonia Stanton ,  Eric Dean Battenberg ,  Russell John Wyatt Skerry-Ryan

IPC: G06N3/088 ,  G06N3/0455

Abstract: Provided is a noisy channel generative model of two sequences, for example text and speech, which enables uncovering the associations between the two modalities when limited paired data is available. To address the intractability of the exact model under a realistic data set-up, example aspects of the present disclosure include a variational inference approximation. To train this variational model with categorical data, a KL encoder loss approach is proposed which has connections to the wake-sleep algorithm.