公开(公告)号：US20220405046A1

公开(公告)日：2022-12-22

申请号：US17812320

申请日：2022-07-13

Applicant: Google LLC

Inventor： Brandon M. Ballinger ,  Johan Schalkwyk ,  Michael H. Cohen ,  William J. Byrne ,  Gudmundur Hafsteinsson ,  Michael J. Lebeau

IPC: G06F3/16 ,  G06F40/284 ,  G06F40/58 ,  G06F3/04886 ,  G10L15/00 ,  G10L15/18 ,  G10L15/183 ,  G10L15/22 ,  G10L15/26 ,  G10L15/30

Abstract: A computer-implemented input-method editor process includes receiving a request from a user for an application-independent input method editor having written and spoken input capabilities, identifying that the user is about to provide spoken input to the application-independent input method editor, and receiving a spoken input from the user. The spoken input corresponds to input to an application and is converted to text that represents the spoken input. The text is provided as input to the application.

公开(公告)号：US20220157312A1

公开(公告)日：2022-05-19

申请号：US17650173

申请日：2022-02-07

Applicant: Google LLC

Inventor： Alexander H. Gruenstein ,  Johan Schalkwyk ,  Matthew Sharifi

IPC: G10L15/22 ,  G06F3/16 ,  G10L15/08 ,  G10L15/30 ,  G10L25/51

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for hotword trigger suppression are disclosed. In one aspect, a method includes the actions of receiving, by a microphone of a computing device, audio corresponding to playback of an item of media content, the audio including an utterance of a predefined hotword that is associated with performing an operation on the computing device. The actions further include processing the audio. The actions further include in response to processing the audio, suppressing performance of the operation on the computing device.

公开(公告)号：US11257498B2

公开(公告)日：2022-02-22

申请号：US17100109

申请日：2020-11-20

Applicant: Google LLC

Inventor： Alexander H. Gruenstein ,  Johan Schalkwyk ,  Matthew Sharifi

IPC: G10L15/22 ,  G06F3/16 ,  G10L15/08 ,  G10L15/30 ,  G10L25/51 ,  G10L17/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for hotword trigger suppression are disclosed. In one aspect, a method includes the actions of receiving, by a microphone of a computing device, audio corresponding to playback of an item of media content, the audio including an utterance of a predefined hotword that is associated with performing an operation on the computing device. The actions further include processing the audio. The actions further include in response to processing the audio, suppressing performance of the operation on the computing device.

公开(公告)号：US20220005458A1

公开(公告)日：2022-01-06

申请号：US17479285

申请日：2021-09-20

Applicant: Google LLC

Inventor： Françoise Beaufays ,  Johan Schalkwyk ,  Khe Chai Sim

IPC: G10L13/047 ,  G10L15/06

Abstract: Processor(s) of a client device can: identify a textual segment stored locally at the client device; process the textual segment, using a speech synthesis model stored locally at the client device, to generate synthesized speech audio data that includes synthesized speech of the identified textual segment; process the synthesized speech, using an on-device speech recognition model that is stored locally at the client device, to generate predicted output; and generate a gradient based on comparing the predicted output to ground truth output that corresponds to the textual segment. In some implementations, the generated gradient is used, by processor(s) of the client device, to update weights of the on-device speech recognition model. In some implementations, the generated gradient is additionally or alternatively transmitted to a remote system for use in remote updating of global weights of a global speech recognition model.

公开(公告)号：US20210405868A1

公开(公告)日：2021-12-30

申请号：US17469622

申请日：2021-09-08

Applicant: Google LLC

Inventor： Shumin Zhai ,  Thomas Breuel ,  Ouais Alsharif ,  Yu Ouyang ,  Francoise Beaufays ,  Johan Schalkwyk

IPC: G06F3/0488 ,  G06N3/04 ,  G06F40/232 ,  G06F40/274 ,  G06F40/279 ,  G06F3/02 ,  G06F3/023 ,  G06F3/0489 ,  G06F3/0482 ,  G06N3/08

Abstract: In some examples, a computing device includes at least one processor; and at least one module, operable by the at least one processor to: output, for display at an output device, a graphical keyboard; receive an indication of a gesture detected at a location of a presence-sensitive input device, wherein the location of the presence-sensitive input device corresponds to a location of the output device that outputs the graphical keyboard; determine, based on at least one spatial feature of the gesture that is processed by the computing device using a neural network, at least one character string, wherein the at least one spatial feature indicates at least one physical property of the gesture; and output, for display at the output device, based at least in part on the processing of the at least one spatial feature of the gesture using the neural network, the at least one character string.

公开(公告)号：US20210334070A1

公开(公告)日：2021-10-28

申请号：US17370656

申请日：2021-07-08

Applicant: GOOGLE LLC

Inventor： Yuan Yuan ,  Johan Schalkwyk ,  Kenneth Mixter

IPC: G06F3/16 ,  G06K9/00 ,  G10L15/22 ,  H04N7/14 ,  H04N7/15

Abstract: The various implementations described herein include methods, devices, and systems for attending to a presenting user. In one aspect, a method is performed at an electronic device that includes an image sensor, microphones, a display, processor(s), and memory. The device (1) obtains audio signals by concurrently receiving audio data at each microphone; (2) determines based on the obtained audio signals that a person is speaking in a vicinity of the device; (3) obtains video data from the image sensor; (4) determines via the video data that the person is not within a field of view of the image sensor; (5) reorients the electronic device based on differences in the received audio data; (6) after reorienting the electronic device, obtains second video data from the image sensor and determines that the person is within the field of view; and (7) attends to the person by directing the display toward the person.

公开(公告)号：US20210327410A1

公开(公告)日：2021-10-21

申请号：US17250165

申请日：2019-10-11

Applicant: Google LLC

Inventor： Françoise Beaufays ,  Johan Schalkwyk ,  Giovanni Motta

IPC: G10L15/00 ,  G10L25/51 ,  G06F3/0484 ,  G06F3/0488

Abstract: Processor(s) of a client device can: receive audio data that captures a spoken utterance of a user of the client device; process, using an on-device speech recognition model, the audio data to generate a predicted textual segment that is a prediction of the If spoken utterance; cause at least part of the predicted textual segment to be rendered (e.g., visually and/or audibly); receive further user interface input that is a correction of the predicted textual segment to an alternate textual segment; and generate a gradient based on comparing at least part of the predicted output to ground truth output that corresponds to the alternate textual segment. The gradient is used, by processor(s) of the client device, to update weights of the on-device speech recognition model and/or is transmitted to a remote system for use in remote updating of global weights of a global speech recognition model.

公开(公告)号：US20210104223A1

公开(公告)日：2021-04-08

申请号：US16959546

申请日：2019-10-02

Applicant: Google LLC

Inventor： Françoise Beaufays ,  Johan Schalkwyk ,  Khe Chai Sim

IPC: G10L13/047 ,  G10L15/06

Abstract: Processor(s) of a client device can: identify a textual segment stored locally at the client device; process the textual segment, using a speech synthesis model stored locally at the client device, to generate synthesized speech audio data that includes synthesized speech of the identified textual segment; process the synthesized speech, using an on-device speech recognition model that is stored locally at the client device, to generate predicted output; and generate a gradient based on comparing the predicted output to ground truth output that corresponds to the textual segment. In some implementations, the generated gradient is used, by processor(s) of the client device, to update weights of the on-device speech recognition model. In some implementations, the generated gradient is additionally or alternatively transmitted to a remote system for use in remote updating of global weights of a global speech recognition model.

公开(公告)号：US20200293276A1

公开(公告)日：2020-09-17

申请号：US16892749

申请日：2020-06-04

Applicant: Google LLC

Inventor： Brandon M. Ballinger ,  Johan Schalkwyk ,  Michael H. Cohen ,  William J. Byrne ,  Gudmundur Hafsteinsson ,  Michael J. Lebeau

IPC: G06F3/16 ,  G10L15/183 ,  G10L15/26 ,  G10L15/30 ,  G06F40/58 ,  G06F40/284 ,  G10L15/18 ,  G06F3/0488 ,  G10L15/00 ,  G10L15/22

Abstract: A computer-implemented input-method editor process includes receiving a request from a user for an application-independent input method editor having written and spoken input capabilities, identifying that the user is about to provide spoken input to the application-independent input method editor, and receiving a spoken input from the user. The spoken input corresponds to input to an application and is converted to text that represents the spoken input. The text is provided as input to the application.

公开(公告)号：US10714096B2

公开(公告)日：2020-07-14

申请号：US15981376

申请日：2018-05-16

Applicant: Google LLC

Inventor： Andrew Rubin ,  Johan Schalkwyk ,  Maria Carolina Parada San Martin

IPC: G10L15/00 ,  G10L17/24 ,  G06F21/32 ,  G10L25/51 ,  G06F21/46 ,  G10L15/22 ,  G10L15/06 ,  G10L15/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining hotword suitability. In one aspect, a method includes receiving speech data that encodes a candidate hotword spoken by a user, evaluating the speech data or a transcription of the candidate hotword, using one or more predetermined criteria, generating a hotword suitability score for the candidate hotword based on evaluating the speech data or a transcription of the candidate hotword, using one or more predetermined criteria, and providing a representation of the hotword suitability score for display to the user.