-
公开(公告)号:US20250069588A1
公开(公告)日:2025-02-27
申请号:US18947557
申请日:2024-11-14
Applicant: GOOGLE LLC
Inventor: Françoise Beaufays , Johan Schalkwyk , Giovanni Motta
IPC: G10L15/00 , G06F3/04842 , G06F3/04883 , G10L25/51
Abstract: Processor(s) of a client device can: receive audio data that captures a spoken utterance of a user of the client device; process, using an on-device speech recognition model, the audio data to generate a predicted textual segment that is a prediction of the spoken utterance; cause at least part of the predicted textual segment to be rendered (e.g., visually and/or audibly); receive further user interface input that is a correction of the predicted textual segment to an alternate textual segment; and generate a gradient based on comparing at least part of the predicted output to ground truth output that corresponds to the alternate textual segment. The gradient is used, by processor(s) of the client device, to update weights of the on-device speech recognition model and/or is transmitted to a remote system for use in remote updating of global weights of a global speech recognition model.
-
公开(公告)号:US20240412734A1
公开(公告)日:2024-12-12
申请号:US18810899
申请日:2024-08-21
Applicant: GOOGLE LLC
Inventor: Alexander H. Gruenstein , Petar Aleksic , Johan Schalkwyk , Pedro J. Moreno Mengibar
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for detecting hotwords using a server. One of the methods includes receiving an audio signal encoding one or more utterances including a first utterance; determining whether at least a portion of the first utterance satisfies a first threshold of being at least a portion of a key phrase; in response to determining that at least the portion of the first utterance satisfies the first threshold of being at least a portion of a key phrase, sending the audio signal to a server system that determines whether the first utterance satisfies a second threshold of being the key phrase, the second threshold being more restrictive than the first threshold; and receiving tagged text data representing the one or more utterances encoded in the audio signal when the server system determines that the first utterance satisfies the second threshold.
-
公开(公告)号:US11817080B2
公开(公告)日:2023-11-14
申请号:US17250165
申请日:2019-10-11
Applicant: Google LLC
Inventor: Françoise Beaufays , Johan Schalkwyk , Giovanni Motta
IPC: G10L15/00 , G06F3/04842 , G06F3/04883 , G10L25/51
CPC classification number: G10L15/00 , G06F3/04842 , G06F3/04883 , G10L25/51
Abstract: Processor(s) of a client device can: receive audio data that captures a spoken utterance of a user of the client device; process, using an on-device speech recognition model, the audio data to generate a predicted textual segment that is a prediction of the spoken utterance; cause at least part of the predicted textual segment to be rendered (e.g., visually and/or audibly); receive further user interface input that is a correction of the predicted textual segment to an alternate textual segment; and generate a gradient based on comparing at least part of the predicted output to ground truth output that corresponds to the alternate textual segment. The gradient is used, by processor(s) of the client device, to update weights of the on-device speech recognition model and/or is transmitted to a remote system for use in remote updating of global weights of a global speech recognition model.
-
公开(公告)号:US20220270590A1
公开(公告)日:2022-08-25
申请号:US16973605
申请日:2020-07-20
Applicant: Google LLC
Inventor: Françoise Beaufays , Khe Chai Sim , Johan Schalkwyk
IPC: G10L15/06 , G10L15/30 , G10L15/22 , G10L15/187
Abstract: Implementations disclosed herein are directed to unsupervised federated training of global machine learning (“ML”) model layers that, after the federated training, can be combined with additional layer(s), thereby resulting in a combined ML model. Processor(s) can: detect audio data that captures a spoken utterance of a user of a client device; process, using a local ML model, the audio data to generate predicted output(s); generate, using unsupervised learning locally at the client device, a gradient based on the predicted output(s); transmit the gradient to a remote system; update weight(s) of the global ML model layers based on the gradient; subsequent to updating the weight(s), train, using supervised learning remotely at the remote system, a combined ML model that includes the updated global ML model layers and additional layer(s); transmit the combined ML model to the client device; and use the combined ML model to make prediction(s) at the client device.
-
公开(公告)号:US11227611B2
公开(公告)日:2022-01-18
申请号:US16891444
申请日:2020-06-03
Applicant: Google LLC
Inventor: Andrew E. Rubin , Johan Schalkwyk , Maria Carolin Parada San Martin
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining hotword suitability. In one aspect, a method includes receiving speech data that encodes a candidate hotword spoken by a user, evaluating the speech data or a transcription of the candidate hotword, using one or more predetermined criteria, generating a hotword suitability score for the candidate hotword based on evaluating the speech data or a transcription of the candidate hotword, using one or more predetermined criteria, and providing a representation of the hotword suitability score for display to the user.
-
公开(公告)号:US11150804B2
公开(公告)日:2021-10-19
申请号:US16862628
申请日:2020-04-30
Applicant: Google LLC
Inventor: Shumin Zhai , Thomas Breuel , Ouais Alsharif , Yu Ouyang , Francoise Beaufays , Johan Schalkwyk
IPC: G06F3/02 , G06F3/0488 , G06N3/04 , G06F40/232 , G06F40/274 , G06F40/279 , G06F3/023 , G06F3/0489 , G06F3/0482 , G06N3/08
Abstract: In some examples, a computing device includes at least one processor; and at least one module, operable by the at least one processor to: output, for display at an output device, a graphical keyboard; receive an indication of a gesture detected at a location of a presence-sensitive input device, wherein the location of the presence-sensitive input device corresponds to a location of the output device that outputs the graphical keyboard; determine, based on at least one spatial feature of the gesture that is processed by the computing device using a neural network, at least one character string, wherein the at least one spatial feature indicates at least one physical property of the gesture; and output, for display at the output device, based at least in part on the processing of the at least one spatial feature of the gesture using the neural network, the at least one character string.
-
公开(公告)号:US20210287678A1
公开(公告)日:2021-09-16
申请号:US17337182
申请日:2021-06-02
Applicant: GOOGLE LLC
Inventor: Alexander H. Gruenstein , Petar Aleksic , Johan Schalkwyk , Pedro J. Moreno Mengibar
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for detecting hotwords using a server. One of the methods includes receiving an audio signal encoding one or more utterances including a first utterance; determining whether at least a portion of the first utterance satisfies a first threshold of being at least a portion of a key phrase; in response to determining that at least the portion of the first utterance satisfies the first threshold of being at least a portion of a key phrase, sending the audio signal to a server system that determines whether the first utterance satisfies a second threshold of being the key phrase, the second threshold being more restrictive than the first threshold; and receiving tagged text data representing the one or more utterances encoded in the audio signal when the server system determines that the first utterance satisfies the second threshold.
-
公开(公告)号:US10002613B2
公开(公告)日:2018-06-19
申请号:US15002044
申请日:2016-01-20
Applicant: Google LLC
Inventor: Andrew E. Rubin , Johan Schalkwyk , Maria Carolina Parada San Martin
IPC: G10L15/00 , G10L21/00 , G10L25/00 , G10L17/24 , G10L25/51 , G06F21/46 , G06F21/32 , G10L15/08 , G10L15/22
CPC classification number: G10L17/24 , G06F21/32 , G06F21/46 , G10L15/06 , G10L15/08 , G10L15/22 , G10L25/51 , G10L2015/0638 , G10L2015/088 , G10L2015/225
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining hotword suitability. In one aspect, a method includes receiving speech data that encodes a candidate hotword spoken by a user, evaluating the speech data or a transcription of the candidate hotword, using one or more predetermined criteria, generating a hotword suitability score for the candidate hotword based on evaluating the speech data or a transcription of the candidate hotword, using one or more predetermined criteria, and providing a representation of the hotword suitability score for display to the user.
-
公开(公告)号:US20250166618A1
公开(公告)日:2025-05-22
申请号:US19029358
申请日:2025-01-17
Applicant: GOOGLE LLC
Inventor: Fadi Biadsy , Johan Schalkwyk , Jason Pelecanos
Abstract: Implementations disclosed herein are directed to techniques for selectively enabling and/or disabling non-transient storage of one or more instances of assistant interaction data for turn(s) of a dialog between a user and an automated assistant. Implementations are additionally or alternatively directed to techniques for retroactive wiping of non-transiently stored assistant interaction data from previous assistant interaction(s).
-
10.
公开(公告)号:US20240290317A1
公开(公告)日:2024-08-29
申请号:US18656197
申请日:2024-05-06
Applicant: GOOGLE LLC
Inventor: Françoise Beaufays , Johan Schalkwyk , Khe Chai Sim
IPC: G10L13/047 , G10L15/06
CPC classification number: G10L13/047 , G10L15/063 , G10L2015/0635
Abstract: Processor(s) of a client device can: identify a textual segment stored locally at the client device; process the textual segment, using a speech synthesis model stored locally at the client device, to generate synthesized speech audio data that includes synthesized speech of the identified textual segment; process the synthesized speech, using an on-device speech recognition model that is stored locally at the client device, to generate predicted output; and generate a gradient based on comparing the predicted output to ground truth output that corresponds to the textual segment. In some implementations, the generated gradient is used, by processor(s) of the client device, to update weights of the on-device speech recognition model. In some implementations, the generated gradient is additionally or alternatively transmitted to a remote system for use in remote updating of global weights of a global speech recognition model.
-
-
-
-
-
-
-
-
-