公开(公告)号：US11715462B2

公开(公告)日：2023-08-01

申请号：US17244891

申请日：2021-04-29

Applicant: MICROSOFT TECHNOLOGY LICENSING, LLC

Inventor： Yu Wu ,  Jinyu Li ,  Shujie Liu ,  Xie Chen ,  Chengyi Wang

IPC: G10L15/16 ,  G06N3/08 ,  G10L15/06 ,  G10L15/22 ,  G06N3/044

CPC classification number: G10L15/16 ,  G06N3/044 ,  G06N3/08 ,  G10L15/063 ,  G10L15/22

Abstract: A computing system is configured to generate a transformer-transducer-based deep neural network. The transformer-transducer-based deep neural network comprises a transformer encoder network and a transducer predictor network. The transformer encoder network has a plurality of layers, each of which includes a multi-head attention network sublayer and a feed-forward network sublayer. The computing system trains an end-to-end (E2E) automatic speech recognition (ASR) model, using the transformer-transducer-based deep neural network. The E2E ASR model has one or more adjustable hyperparameters that are configured to dynamically adjust an efficiency or a performance of E2E ASR model when the E2E ASR model is deployed onto a device or executed by the device.

公开(公告)号：US12217745B2

公开(公告)日：2025-02-04

申请号：US18217888

申请日：2023-07-03

Applicant: Microsoft Technology Licensing, LLC

Inventor： Yao Qian ,  Yu Wu ,  Kenichi Kumatani ,  Shujie Liu ,  Furu Wei ,  Nanshan Zeng ,  Xuedong David Huang ,  Chengyi Wang

IPC: G10L15/187 ,  G06N20/00 ,  G10L15/02 ,  G10L15/06 ,  G10L15/22

Abstract: A system obtains a first training data set comprising labeled speech data or both labeled and unlabeled data corresponding to a high-resource data set as well as latent speech representations based on the first training data set. The system trains a machine learning model on the first training data set to learn phonetically aware speech representations corresponding to the first training data set. The system applies the latent speech representations to a transformer context network to generate contextual representations. The system aligns each of the contextual representations with a phoneme label to generate phonetically-aware contextual representations. The system causes a refinement engine to further refine the machine learning model.

公开(公告)号：US12020694B2

公开(公告)日：2024-06-25

申请号：US18331742

申请日：2023-06-08

Applicant: Microsoft Technology Licensing, LLC

Inventor： Yu Wu ,  Jinyu Li ,  Shujie Liu ,  Xie Chen ,  Chengyi Wang

IPC: G10L15/16 ,  G06N3/044 ,  G06N3/08 ,  G10L15/06 ,  G10L15/22

CPC classification number: G10L15/16 ,  G06N3/044 ,  G06N3/08 ,  G10L15/063 ,  G10L15/22

Abstract: The computing system trains an end-to-end (E2E) automatic speech recognition (ASR) model, using a transformer-transducer-based deep neural network that comprises a transformer encoder network and a transducer predictor network. The E2E ASR model is trained to have one or more adjustable hyperparameters that are configured to dynamically adjust an efficiency or a performance of the E2E ASR model when the E2E ASR model is deployed onto a device or executed by the device, by identifying one or more conditions of the device associated with computational power of the device and setting at least one of the one or more adjustable hyperparameters based on one or more conditions of the device.

公开(公告)号：US11735171B2

公开(公告)日：2023-08-22

申请号：US17320496

申请日：2021-05-14

Applicant: MICROSOFT TECHNOLOGY LICENSING, LLC

Inventor： Yao Qian ,  Yu Wu ,  Kenichi Kumatani ,  Shujie Liu ,  Furu Wei ,  Nanshan Zeng ,  Xuedong David Huang ,  Chengyi Wang

IPC: G10L15/187 ,  G06N20/00 ,  G10L15/06 ,  G10L15/22 ,  G10L15/02

CPC classification number: G10L15/187 ,  G06N20/00 ,  G10L15/02 ,  G10L15/063 ,  G10L15/22 ,  G10L2015/025

Abstract: Systems and methods are provided for training a machine learning model to learn speech representations. Labeled speech data or both labeled and unlabeled data sets is applied to a feature extractor of a machine learning model to generate latent speech representations. The latent speech representations are applied to a quantizer to generate quantized latent speech representations and to a transformer context network to generate contextual representations. Each contextual representation included in the contextual representations is aligned with a phoneme label to generate phonetically-aware contextual representations. Quantized latent representations are aligned with phoneme labels to generate phonetically aware latent speech representations. Systems and methods also include randomly replacing a sub-set of the contextual representations with quantized latent speech representations during their alignments to phoneme labels and aligning the phonetically aware latent speech representations to the contextual representations using supervised learning.