Generating Diverse and Natural Text-To-Speech Samples

    公开(公告)号:US20220246132A1

    公开(公告)日:2022-08-04

    申请号:US17163007

    申请日:2021-01-29

    Applicant: Google LLC

    Abstract: A method of generating diverse and natural text-to-speech (TTS) samples includes receiving a text and generating a speech sample based on the text using a TTS model. A training process trains the TTS model to generate the speech sample by receiving training samples. Each training sample includes a spectrogram and a training text corresponding to the spectrogram. For each training sample, the training process identifies speech units associated with the training text. For each speech unit, the training process generates a speech embedding, aligns the speech embedding with a portion of the spectrogram, extracts a latent feature from the aligned portion of the spectrogram, and assigns a quantized embedding to the latent feature. The training process generates the speech sample by decoding a concatenation of the speech embeddings and a quantized embeddings for the speech units associated with the training text corresponding to the spectrogram.

    SYSTEMS AND METHODS FOR NEAREST-NEIGHBOR PREDICTION BASED MACHINE LEARNED MODELS

    公开(公告)号:US20220245917A1

    公开(公告)日:2022-08-04

    申请号:US17559633

    申请日:2021-12-22

    Applicant: Google LLC

    Abstract: Systems and methods of the present disclosure can include a computer-implemented method. The method can include obtaining a machine-learned model comprising one or more layers. At least a first layer of the one or more layers can be configured to receive a set of query vectors respectively associated with layer inputs, determine similarity measures the key vectors and the query vectors, apply a normalization operation to the plurality of respective similarity measures, and determine an output based on the normalized respective similarity measures and a plurality of class labels respectively associated with the plurality of key vectors.

    Two-level text-to-speech systems using synthetic training data

    公开(公告)号:US12260851B2

    公开(公告)日:2025-03-25

    申请号:US17305809

    申请日:2021-07-14

    Applicant: Google LLC

    Abstract: A method includes obtaining training data including a plurality of training audio signals and corresponding transcripts. Each training audio signal is spoken by a target speaker in a first accent/dialect. For each training audio signal of the training data, the method includes generating a training synthesized speech representation spoken by the target speaker in a second accent/dialect different than the first accent/dialect and training a text-to-speech (TTS) system based on the corresponding transcript and the training synthesized speech representation. The method also includes receiving an input text utterance to be synthesized into speech in the second accent/dialect. The method also includes obtaining conditioning inputs that include a speaker embedding and an accent/dialect identifier that identifies the second accent/dialect. The method also includes generating an output audio waveform corresponding to a synthesized speech representation of the input text sequence that clones the voice of the target speaker in the second accent/dialect.

    Phonemes And Graphemes for Neural Text-to-Speech

    公开(公告)号:US20240339106A1

    公开(公告)日:2024-10-10

    申请号:US18746809

    申请日:2024-06-18

    Applicant: Google LLC

    CPC classification number: G10L13/086 G06F40/263 G06F40/279 G06N3/08 G10L13/047

    Abstract: A method includes receiving a text input including a sequence of words represented as an input encoder embedding. The input encoder embedding includes a plurality of tokens, with the plurality of tokens including a first set of grapheme tokens representing the text input as respective graphemes and a second set of phoneme tokens representing the text input as respective phonemes. The method also includes, for each respective phoneme token of the second set of phoneme tokens: identifying a respective word of the sequence of words corresponding to the respective phoneme token and determining a respective grapheme token representing the respective word of the sequence of words corresponding to the respective phoneme token. The method also includes generating an output encoder embedding based on a relationship between each respective phoneme token and the corresponding grapheme token determined to represent a same respective word as the respective phoneme token.

    Phonemes and graphemes for neural text-to-speech

    公开(公告)号:US12020685B2

    公开(公告)日:2024-06-25

    申请号:US17643684

    申请日:2021-12-10

    Applicant: Google LLC

    CPC classification number: G10L13/086 G06F40/263 G06F40/279 G06N3/08 G10L13/047

    Abstract: A method includes receiving a text input including a sequence of words represented as an input encoder embedding. The input encoder embedding includes a plurality of tokens, with the plurality of tokens including a first set of grapheme tokens representing the text input as respective graphemes and a second set of phoneme tokens representing the text input as respective phonemes. The method also includes, for each respective phoneme token of the second set of phoneme tokens: identifying a respective word of the sequence of words corresponding to the respective phoneme token and determining a respective grapheme token representing the respective word of the sequence of words corresponding to the respective phoneme token. The method also includes generating an output encoder embedding based on a relationship between each respective phoneme token and the corresponding grapheme token determined to represent a same respective word as the respective phoneme token.

    Two-Level Speech Prosody Transfer
    29.
    发明申请

    公开(公告)号:US20230064749A1

    公开(公告)日:2023-03-02

    申请号:US18054604

    申请日:2022-11-11

    Applicant: Google LLC

    Abstract: A method includes receiving an input text utterance to be synthesized into expressive speech having an intended prosody and a target voice and generating, using a first text-to-speech (TTS) model, an intermediate synthesized speech representation for the input text utterance. The intermediate synthesized speech representation possesses the intended prosody. The method also includes providing the intermediate synthesized speech representation to a second TTS model that includes an encoder portion and a decoder portion. The encoder portion is configured to encode the intermediate synthesized speech representation into an utterance embedding that specifies the intended prosody. The decoder portion is configured to process the input text utterance and the utterance embedding to generate an output audio signal of expressive speech that has the intended prosody specified by the utterance embedding and speaker characteristics of the target voice.

Patent Agency Ranking