公开(公告)号：US20240160403A1

公开(公告)日：2024-05-16

申请号：US18421189

申请日：2024-01-24

Applicant: Google LLC

Inventor： Brandon M. Ballinger ,  Johan Schalkwyk ,  Michael H. Cohen ,  William J. Byrne ,  Gudmundur Hafsteinsson ,  Michael J. Lebeau

IPC: G06F3/16 ,  G06F3/04886 ,  G06F40/284 ,  G06F40/58 ,  G10L15/00 ,  G10L15/18 ,  G10L15/183 ,  G10L15/22 ,  G10L15/26 ,  G10L15/30

CPC classification number: G06F3/167 ,  G06F3/04886 ,  G06F40/284 ,  G06F40/58 ,  G10L15/005 ,  G10L15/18 ,  G10L15/183 ,  G10L15/22 ,  G10L15/26 ,  G10L15/30 ,  G10L15/197

Abstract: A computer-implemented input-method editor process includes receiving a request from a user for an application-independent input method editor having written and spoken input capabilities, identifying that the user is about to provide spoken input to the application-independent input method editor, and receiving a spoken input from the user. The spoken input corresponds to input to an application and is converted to text that represents the spoken input. The text is provided as input to the application.

公开(公告)号：US20230343340A1

公开(公告)日：2023-10-26

申请号：US18345077

申请日：2023-06-30

Applicant: GOOGLE LLC

Inventor： Alexander H. Gruenstein ,  Petar Aleksic ,  Johan Schalkwyk ,  Pedro J. Moreno Mengibar

IPC: G10L15/30 ,  G10L15/32 ,  G10L15/26

CPC classification number: G10L15/30 ,  G10L15/32 ,  G10L15/26 ,  G10L15/183

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for detecting hotwords using a server. One of the methods includes receiving an audio signal encoding one or more utterances including a first utterance; determining whether at least a portion of the first utterance satisfies a first threshold of being at least a portion of a key phrase; in response to determining that at least the portion of the first utterance satisfies the first threshold of being at least a portion of a key phrase, sending the audio signal to a server system that determines whether the first utterance satisfies a second threshold of being the key phrase, the second threshold being more restrictive than the first threshold; and receiving tagged text data representing the one or more utterances encoded in the audio signal when the server system determines that the first utterance satisfies the second threshold.

公开(公告)号：US11127392B2

公开(公告)日：2021-09-21

申请号：US16959546

申请日：2019-10-02

Applicant: Google LLC

Inventor： Françoise Beaufays ,  Johan Schalkwyk ,  Khe Chai Sim

IPC: G10L13/00 ,  G10L13/047 ,  G10L15/06

Abstract: Processor(s) of a client device can: identify a textual segment stored locally at the client device; process the textual segment, using a speech synthesis model stored locally at the client device, to generate synthesized speech audio data that includes synthesized speech of the identified textual segment; process the synthesized speech, using an on-device speech recognition model that is stored locally at the client device, to generate predicted output; and generate a gradient based on comparing the predicted output to ground truth output that corresponds to the textual segment. In some implementations, the generated gradient is used, by processor(s) of the client device, to update weights of the on-device speech recognition model. In some implementations, the generated gradient is additionally or alternatively transmitted to a remote system for use in remote updating of global weights of a global speech recognition model.

公开(公告)号：US20200302941A1

公开(公告)日：2020-09-24

申请号：US16891444

申请日：2020-06-03

Applicant: Google LLC

Inventor： Andrew E Rubin ,  Johan Schalkwyk ,  Maria Carolin Parada San Martin

IPC: G10L17/24 ,  G06F21/32 ,  G10L25/51 ,  G06F21/46 ,  G10L15/22 ,  G10L15/06 ,  G10L15/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining hotword suitability. In one aspect, a method includes receiving speech data that encodes a candidate hotword spoken by a user, evaluating the speech data or a transcription of the candidate hotword, using one or more predetermined criteria, generating a hotword suitability score for the candidate hotword based on evaluating the speech data or a transcription of the candidate hotword, using one or more predetermined criteria, and providing a representation of the hotword suitability score for display to the user.

公开(公告)号：US20200257447A1

公开(公告)日：2020-08-13

申请号：US16862628

申请日：2020-04-30

Applicant: Google LLC

Inventor： Shumin Zhai ,  Thomas Breuel ,  Ouais Alsharif ,  Yu Ouyang ,  Francoise Beaufays ,  Johan Schalkwyk

IPC: G06F3/0488 ,  G06N3/04 ,  G06F40/232 ,  G06F40/274 ,  G06F40/279 ,  G06F3/02 ,  G06F3/023 ,  G06F3/0489 ,  G06F3/0482 ,  G06N3/08

Abstract: In some examples, a computing device includes at least one processor; and at least one module, operable by the at least one processor to: output, for display at an output device, a graphical keyboard; receive an indication of a gesture detected at a location of a presence-sensitive input device, wherein the location of the presence-sensitive input device corresponds to a location of the output device that outputs the graphical keyboard; determine, based on at least one spatial feature of the gesture that is processed by the computing device using a neural network, at least one character string, wherein the at least one spatial feature indicates at least one physical property of the gesture; and output, for display at the output device, based at least in part on the processing of the at least one spatial feature of the gesture using the neural network, the at least one character string.

公开(公告)号：US20190304465A1

公开(公告)日：2019-10-03

申请号：US16392829

申请日：2019-04-24

Applicant: Google LLC

Inventor： Alexander H. Gruenstein ,  Petar Aleksic ,  Johan Schalkwyk ,  Pedro J. Moreno Mengibar

IPC: G10L15/30 ,  G10L15/32 ,  G10L15/26

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for detecting hotwords using a server. One of the methods includes receiving an audio signal encoding one or more utterances including a first utterance; determining whether at least a portion of the first utterance satisfies a first threshold of being at least a portion of a key phrase; in response to determining that at least the portion of the first utterance satisfies the first threshold of being at least a portion of a key phrase, sending the audio signal to a server system that determines whether the first utterance satisfies a second threshold of being the key phrase, the second threshold being more restrictive than the first threshold; and receiving tagged text data representing the one or more utterances encoded in the audio signal when the server system determines that the first utterance satisfies the second threshold.

公开(公告)号：US20190155504A1

公开(公告)日：2019-05-23

申请号：US16261640

申请日：2019-01-30

Applicant: Google LLC

Inventor： Shumin Zhai ,  Thomas Breuel ,  Ouais Alsharif ,  Yu Ouyang ,  Francoise Beaufays ,  Johan Schalkwyk

IPC: G06F3/0488 ,  G06N3/04 ,  G06F17/27 ,  G06F3/023 ,  G06F3/0489 ,  G06F3/02 ,  G06N3/08 ,  G06F3/0482

Abstract: In some examples, a computing device includes at least one processor; and at least one module, operable by the at least one processor to: output, for display at an output device, a graphical keyboard; receive an indication of a gesture detected at a location of a presence-sensitive input device, wherein the location of the presence-sensitive input device corresponds to a location of the output device that outputs the graphical keyboard; determine, based on at least one spatial feature of the gesture that is processed by the computing device using a neural network, at least one character string, wherein the at least one spatial feature indicates at least one physical property of the gesture; and output, for display at the output device, based at least in part on the processing of the at least one spatial feature of the gesture using the neural network, the at least one character string.

公开(公告)号：US20180336906A1

公开(公告)日：2018-11-22

申请号：US15981376

申请日：2018-05-16

Applicant: Google LLC

Inventor： Andrew Rubin ,  Johan Schalkwyk ,  Maria Carolina Parada San Martin

IPC: G10L17/24 ,  G10L15/08 ,  G06F21/32 ,  G10L15/22

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining hotword suitability. In one aspect, a method includes receiving speech data that encodes a candidate hotword spoken by a user, evaluating the speech data or a transcription of the candidate hotword, using one or more predetermined criteria, generating a hotword suitability score for the candidate hotword based on evaluating the speech data or a transcription of the candidate hotword, using one or more predetermined criteria, and providing a representation of the hotword suitability score for display to the user.

公开(公告)号：US20180130469A1

公开(公告)日：2018-05-10

申请号：US15799501

申请日：2017-10-31

Applicant: Google LLC

Inventor： Alexander H. Gruenstein ,  Johan Schalkwyk ,  Matthew Sharifi

IPC: G10L15/22 ,  G10L15/08 ,  G10L25/51 ,  G10L15/30

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for hotword trigger suppression are disclosed. In one aspect, a method includes the actions of receiving, by a microphone of a computing device, audio corresponding to playback of an item of media content, the audio including an utterance of a predefined hotword that is associated with performing an operation on the computing device. The actions further include processing the audio. The actions further include in response to processing the audio, suppressing performance of the operation on the computing device.

公开(公告)号：US12020703B2

公开(公告)日：2024-06-25

申请号：US17532819

申请日：2021-11-22

Applicant: GOOGLE LLC

Inventor： Jaclyn Konzelmann ,  Trevor Strohman ,  Jonathan Bloom ,  Johan Schalkwyk ,  Joseph Smarr

IPC: G10L15/22 ,  G06N20/00 ,  G08B5/36 ,  G10L15/08 ,  G10L15/18

CPC classification number: G10L15/22 ,  G06N20/00 ,  G08B5/36 ,  G10L15/18 ,  G10L2015/088 ,  G10L2015/223

Abstract: As part of a dialog session between a user and an automated assistant, implementations can process, using a streaming ASR model, a stream of audio data that captures a portion of a spoken utterance to generate ASR output, process, using an NLU model, the ASR output to generate NLU output, and cause, based on the NLU output, a stream of fulfillment data to be generated. Further, implementations can further determine, based on processing the stream of audio data, audio-based characteristics associated with the portion of the spoken utterance captured in the stream of audio data. Based on the audio-based characteristics and/the stream of NLU output, implementations can determine whether the user has paused in providing the spoken utterance or has completed providing of the spoken utterance. If the user has paused, implementations can cause natural conversation output to be provided for presentation to the user.