公开(公告)号：US20210407498A1

公开(公告)日：2021-12-30

申请号：US17474829

申请日：2021-09-14

Applicant: Microsoft Technology Licensing, LLC

Inventor： Emilian Stoimenov ,  Rui Zhao ,  Kaustubh Prakash Kalgaonkar ,  Ivaylo Andreanov Enchev ,  Khuram Shahid ,  Anthony Phillip Stark ,  Guoli Ye ,  Mahadevan Srinivasan ,  Yifan Gong ,  Hosam Adel Khalil

IPC: G10L15/16 ,  G06N3/08 ,  G10L17/24

Abstract: Generally discussed herein are devices, systems, and methods for on-device detection of a wake word. A device can include a memory including model parameters that define a custom wake word detection model, the wake word detection model including a recurrent neural network transducer (RNNT) and a lookup table (LUT), the LUT indicating a hidden vector to be provided in response to a phoneme of a user-specified wake word, a microphone to capture audio, and processing circuitry to receive the audio from the microphone, determine, using the wake word detection model, whether the audio includes an utterance of the user-specified wake word, and wake up a personal assistant after determining the audio includes the utterance of the user-specified wake word.

公开(公告)号：US11798535B2

公开(公告)日：2023-10-24

申请号：US17474829

申请日：2021-09-14

Applicant: Microsoft Technology Licensing, LLC

Inventor： Emilian Stoimenov ,  Rui Zhao ,  Kaustubh Prakash Kalgaonkar ,  Ivaylo Andreanov Enchev ,  Khuram Shahid ,  Anthony Phillip Stark ,  Guoli Ye ,  Mahadevan Srinivasan ,  Yifan Gong ,  Hosam Adel Khalil

IPC: G10L15/16 ,  G06N3/08 ,  G10L17/24 ,  G10L15/08

CPC classification number: G10L15/16 ,  G06N3/08 ,  G10L17/24 ,  G10L2015/088

Abstract: Generally discussed herein are devices, systems, and methods for on-device detection of a wake word. A device can include a memory including model parameters that define a custom wake word detection model, the wake word detection model including a recurrent neural network transducer (RNNT) and a lookup table (LUT), the LUT indicating a hidden vector to be provided in response to a phoneme of a user-specified wake word, a microphone to capture audio, and processing circuitry to receive the audio from the microphone, determine, using the wake word detection model, whether the audio includes an utterance of the user-specified wake word, and wake up a personal assistant after determining the audio includes the utterance of the user-specified wake word.

公开(公告)号：US20210304769A1

公开(公告)日：2021-09-30

申请号：US15931788

申请日：2020-05-14

Applicant: MICROSOFT TECHNOLOGY LICENSING, LLC

Inventor： Guoli Ye ,  Yan Huang ,  Wenning Wei ,  Lei He ,  Eva Sharma ,  Jian Wu ,  Yao Tian ,  Edward C. Lin ,  Yifan Gong ,  Rui Zhao ,  Jinyu Li ,  William Maxwell Gale

IPC: G10L15/26 ,  G10L15/16 ,  G10L15/06 ,  G10L13/08

Abstract: Systems, methods, and devices are provided for generating and using text-to-speech (TTS) data for improved speech recognition models. A main model is trained with keyword independent baseline training data. In some instances, acoustic and language model sub-components of the main model are modified with new TTS training data. In some instances, the new TTS training is obtained from a multi-speaker neural TTS system for a keyword that is underrepresented in the baseline training data. In some instances, the new TTS training data is used for pronunciation learning and normalization of keyword dependent confidence scores in keyword spotting (KWS) applications. In some instances, the new TTS training data is used for rapid speaker adaptation in speech recognition models.

公开(公告)号：US11587569B2

公开(公告)日：2023-02-21

申请号：US15931788

申请日：2020-05-14

Applicant: MICROSOFT TECHNOLOGY LICENSING, LLC

Inventor： Guoli Ye ,  Yan Huang ,  Wenning Wei ,  Lei He ,  Eva Sharma ,  Jian Wu ,  Yao Tian ,  Edward C. Lin ,  Yifan Gong ,  Rui Zhao ,  Jinyu Li ,  William Maxwell Gale

IPC: G10L15/26 ,  G10L13/08 ,  G10L15/06 ,  G10L15/16

Abstract: Systems, methods, and devices are provided for generating and using text-to-speech (TTS) data for improved speech recognition models. A main model is trained with keyword independent baseline training data. In some instances, acoustic and language model sub-components of the main model are modified with new TTS training data. In some instances, the new TTS training is obtained from a multi-speaker neural TTS system for a keyword that is underrepresented in the baseline training data. In some instances, the new TTS training data is used for pronunciation learning and normalization of keyword dependent confidence scores in keyword spotting (KWS) applications. In some instances, the new TTS training data is used for rapid speaker adaptation in speech recognition models.

公开(公告)号：US11132992B2

公开(公告)日：2021-09-28

申请号：US16522416

申请日：2019-07-25

Applicant: Microsoft Technology Licensing, LLC

Inventor： Emilian Stoimenov ,  Rui Zhao ,  Kaustubh Prakash Kalgaonkar ,  Ivaylo Andreanov Enchev ,  Khuram Shahid ,  Anthony Phillip Stark ,  Guoli Ye ,  Mahadevan Srinivasan ,  Yifan Gong ,  Hosam Adel Khalil

IPC: G10L15/00 ,  G10L15/16 ,  G06N3/08 ,  G10L17/24 ,  G10L15/08

Abstract: Generally discussed herein are devices, systems, and methods for on-device detection of a wake word. A device can include a memory including model parameters that define a custom wake word detection model, the wake word detection model including a recurrent neural network transducer (RNNT) and a lookup table (LUT), the LUT indicating a hidden vector to be provided in response to a phoneme of a user-specified wake word, a microphone to capture audio, and processing circuitry to receive the audio from the microphone, determine, using the wake word detection model, whether the audio includes an utterance of the user-specified wake word, and wake up a personal assistant after determining the audio includes the utterance of the user-specified wake word.

公开(公告)号：US12205596B2

公开(公告)日：2025-01-21

申请号：US18108316

申请日：2023-02-10

Applicant: Microsoft Technology Licensing, LLC

Inventor： Guoli Ye ,  Yan Huang ,  Wenning Wei ,  Lei He ,  Eva Sharma ,  Jian Wu ,  Yao Tian ,  Edward C. Lin ,  Yifan Gong ,  Rui Zhao ,  Jinyu Li ,  William Maxwell Gale

IPC: G10L15/26 ,  G10L13/08 ,  G10L15/06 ,  G10L15/16

Abstract: Systems, methods, and devices are provided for generating and using text-to-speech (TTS) data for improved speech recognition models. A main model is trained with keyword independent baseline training data. In some instances, acoustic and language model sub-components of the main model are modified with new TTS training data. In some instances, the new TTS training is obtained from a multi-speaker neural TTS system for a keyword that is underrepresented in the baseline training data. In some instances, the new TTS training data is used for pronunciation learning and normalization of keyword dependent confidence scores in keyword spotting (KWS) applications. In some instances, the new TTS training data is used for rapid speaker adaptation in speech recognition models.

公开(公告)号：US20220254334A1

公开(公告)日：2022-08-11

申请号：US17539622

申请日：2021-12-01

Applicant: Microsoft Technology Licensing, LLC

Inventor： Emilian Stoimenov ,  Khuram Shahid ,  Guoli Ye ,  Hosam Adel Khalil ,  Yifan Gong

IPC: G10L15/07 ,  G10L15/02 ,  G10L15/187 ,  G10L13/033 ,  G10L15/22 ,  G10L13/10 ,  G10L13/00

Abstract: Generally discussed herein are devices, systems, and methods for custom wake word selection assistance. A method can include receiving, at a device, data indicating a custom wake word provided by a user, determining one or more characteristics of the custom wake word, determining that use of the custom wake word will cause more than a threshold rate of false detections based on the characteristics, rejecting the custom wake word as the wake word for accessing a personal assistant in response to determining that use of the custom wake word will cause more than a threshold rate of false detections, and setting the custom wake word as the wake word in response to determining that use of the custom wake word will not cause more than the threshold rate of false detections.

公开(公告)号：US11222622B2

公开(公告)日：2022-01-11

申请号：US16522427

申请日：2019-07-25

Applicant: Microsoft Technology Licensing, LLC

Inventor： Emilian Stoimenov ,  Khuram Shahid ,  Guoli Ye ,  Hosam Adel Khalil ,  Yifan Gong

IPC: G10L15/07 ,  G10L15/02 ,  G10L15/187 ,  G10L13/033 ,  G10L15/22 ,  G10L13/10 ,  G10L13/00 ,  G10L15/08

Abstract: Generally discussed herein are devices, systems, and methods for custom wake word selection assistance. A method can include receiving, at a device, data indicating a custom wake word provided by a user, determining one or more characteristics of the custom wake word, determining that use of the custom wake word will cause more than a threshold rate of false detections based on the characteristics, rejecting the custom wake word as the wake word for accessing a personal assistant in response to determining that use of the custom wake word will cause more than a threshold rate of false detections, and setting the custom wake word as the wake word in response to determining that use of the custom wake word will not cause more than the threshold rate of false detections.

公开(公告)号：US11790891B2

公开(公告)日：2023-10-17

申请号：US17539622

申请日：2021-12-01

Applicant: Microsoft Technology Licensing, LLC

Inventor： Emilian Stoimenov ,  Khuram Shahid ,  Guoli Ye ,  Hosam Adel Khalil ,  Yifan Gong

IPC: G10L15/07 ,  G10L15/02 ,  G10L15/187 ,  G10L13/033 ,  G10L15/22 ,  G10L13/10 ,  G10L13/00 ,  G10L15/08

CPC classification number: G10L15/07 ,  G10L13/00 ,  G10L13/033 ,  G10L13/10 ,  G10L15/02 ,  G10L15/187 ,  G10L15/22 ,  G10L2015/025 ,  G10L2015/088 ,  G10L2015/223

Abstract: Generally discussed herein are devices, systems, and methods for custom wake word selection assistance. A method can include receiving, at a device, data indicating a custom wake word provided by a user, determining one or more characteristics of the custom wake word, determining that use of the custom wake word will cause more than a threshold rate of false detections based on the characteristics, rejecting the custom wake word as the wake word for accessing a personal assistant in response to determining that use of the custom wake word will cause more than a threshold rate of false detections, and setting the custom wake word as the wake word in response to determining that use of the custom wake word will not cause more than the threshold rate of false detections.

公开(公告)号：US10964309B2

公开(公告)日：2021-03-30

申请号：US16410556

申请日：2019-05-13

Applicant: Microsoft Technology Licensing, LLC

Inventor： Jinyu Li ,  Guoli Ye ,  Rui Zhao ,  Yifan Gong ,  Ke Li

IPC: G10L15/00 ,  G10L15/06 ,  G10L15/04

Abstract: A CS CTC model may be initialed from a major language CTC model by keeping network hidden weights and replacing output tokens with a union of major and secondary language output tokens. The initialized model may be trained by updating parameters with training data from both languages, and a LID model may also be trained with the data. During a decoding process for each of a series of audio frames, if silence dominates a current frame then a silence output token may be emitted. If silence does not dominate the frame, then a major language output token posterior vector from the CS CTC model may be multiplied with the LID major language probability to create a probability vector from the major language. A similar step is performed for the secondary language, and the system may emit an output token associated with the highest probability across all tokens from both languages.