公开(公告)号：US11698699B2

公开(公告)日：2023-07-11

申请号：US17735937

申请日：2022-05-03

Applicant: Google LLC

Inventor： Xu Liu ,  Shumin Zhai ,  Adrian Pittman ,  Billy Dou

IPC: G06F3/048 ,  G06F3/041 ,  G06F3/04886

CPC classification number: G06F3/04186 ,  G06F3/04886

Abstract: A method includes detecting a touch input by a user at a position on a virtual keyboard. The method further includes applying a spatial model to map the touch input to one or more first candidate keys of the virtual keyboard, wherein the spatial model maps the touch input to the one or more first candidate keys based on a heatmap of prior touch inputs by the user. The method further includes applying a language model to map the touch input to one or more second candidate keys of the virtual keyboard, wherein the language model maps the touch input to the one or more second candidate keys based on comparing the position of the touch input to one or more predicted next keys. The method further includes selecting a particular key of the one or more first and second candidate keys as indicated by the touch input.

公开(公告)号：US11347352B2

公开(公告)日：2022-05-31

申请号：US17079492

申请日：2020-10-25

Applicant: Google LLC

Inventor： Xu Liu ,  Shumin Zhai ,  Adrian Pittman ,  Billy Dou

IPC: G06F3/048 ,  G06F3/041 ,  G06F3/04886

Abstract: A method includes detecting a touch input by a user at a position on a virtual keyboard. The method further includes applying a spatial model to map the touch input to one or more first candidate keys of the virtual keyboard, wherein the spatial model maps the touch input to the one or more first candidate keys based on a heatmap of prior touch inputs by the user. The method further includes applying a language model to map the touch input to one or more second candidate keys of the virtual keyboard, wherein the language model maps the touch input to the one or more second candidate keys based on comparing the position of the touch input to one or more predicted next keys. The method further includes selecting a particular key of the one or more first and second candidate keys as indicated by the touch input.

公开(公告)号：US11334717B2

公开(公告)日：2022-05-17

申请号：US16683895

申请日：2019-11-14

Applicant: Google LLC

Inventor： Shumin Zhai ,  Ciprian Ioan Chelba

IPC: G06F40/284 ,  G06F3/04886 ,  G06F3/023 ,  G06F3/04883 ,  H04L51/02 ,  G06F40/242 ,  G06F40/263 ,  H04L51/063

Abstract: A computing device outputs for display at a presence-sensitive display, a graphical keyboard comprising a plurality of keys, receives an indication of at least one gesture to select a group of keys of the plurality of keys, and determines at least one characteristic associated with the at least one gesture to select the group of keys of the plurality of keys. The computing device modifies a spatial model based at least in part on the at least one characteristic and determines a candidate word based at least in part on data provided by the spatial model and a language model, wherein the spatial model provides data based at least in part on the indication of the at least one gesture and wherein the language model provides data based at least in part on a lexicon. The computing device outputs for display at the presence-sensitive display, the candidate word.

公开(公告)号：US20200257447A1

公开(公告)日：2020-08-13

申请号：US16862628

申请日：2020-04-30

Applicant: Google LLC

Inventor： Shumin Zhai ,  Thomas Breuel ,  Ouais Alsharif ,  Yu Ouyang ,  Francoise Beaufays ,  Johan Schalkwyk

IPC: G06F3/0488 ,  G06N3/04 ,  G06F40/232 ,  G06F40/274 ,  G06F40/279 ,  G06F3/02 ,  G06F3/023 ,  G06F3/0489 ,  G06F3/0482 ,  G06N3/08

Abstract: In some examples, a computing device includes at least one processor; and at least one module, operable by the at least one processor to: output, for display at an output device, a graphical keyboard; receive an indication of a gesture detected at a location of a presence-sensitive input device, wherein the location of the presence-sensitive input device corresponds to a location of the output device that outputs the graphical keyboard; determine, based on at least one spatial feature of the gesture that is processed by the computing device using a neural network, at least one character string, wherein the at least one spatial feature indicates at least one physical property of the gesture; and output, for display at the output device, based at least in part on the processing of the at least one spatial feature of the gesture using the neural network, the at least one character string.

公开(公告)号：US20190155504A1

公开(公告)日：2019-05-23

申请号：US16261640

申请日：2019-01-30

Applicant: Google LLC

Inventor： Shumin Zhai ,  Thomas Breuel ,  Ouais Alsharif ,  Yu Ouyang ,  Francoise Beaufays ,  Johan Schalkwyk

IPC: G06F3/0488 ,  G06N3/04 ,  G06F17/27 ,  G06F3/023 ,  G06F3/0489 ,  G06F3/02 ,  G06N3/08 ,  G06F3/0482

Abstract: In some examples, a computing device includes at least one processor; and at least one module, operable by the at least one processor to: output, for display at an output device, a graphical keyboard; receive an indication of a gesture detected at a location of a presence-sensitive input device, wherein the location of the presence-sensitive input device corresponds to a location of the output device that outputs the graphical keyboard; determine, based on at least one spatial feature of the gesture that is processed by the computing device using a neural network, at least one character string, wherein the at least one spatial feature indicates at least one physical property of the gesture; and output, for display at the output device, based at least in part on the processing of the at least one spatial feature of the gesture using the neural network, the at least one character string.

公开(公告)号：US20180046612A1

公开(公告)日：2018-02-15

申请号：US15795955

申请日：2017-10-27

Applicant: Google LLC

Inventor： Shumin Zhai ,  Ciprian Ioan Chelba

IPC: G06F17/27 ,  G06F3/0488 ,  G06F3/023

CPC classification number: G06F17/277 ,  G06F3/0237 ,  G06F3/04883 ,  G06F3/04886 ,  G06F17/2735 ,  G06F17/275 ,  H04L51/02 ,  H04L51/063

Abstract: A computing device outputs for display at a presence-sensitive display, a graphical keyboard comprising a plurality of keys, receives an indication of at least one gesture to select a group of keys of the plurality of keys, and determines at least one characteristic associated with the at least one gesture to select the group of keys of the plurality of keys. The computing device modifies a spatial model based at least in part on the at least one characteristic and determines a candidate word based at least in part on data provided by the spatial model and a language model, wherein the spatial model provides data based at least in part on the indication of the at least one gesture and wherein the language model provides data based at least in part on a lexicon. The computing device outputs for display at the presence-sensitive display, the candidate word.

公开(公告)号：US20240256047A1

公开(公告)日：2024-08-01

申请号：US18040544

申请日：2020-08-25

Applicant: Google LLC

Inventor： Xuelin Huang ,  Shumin Zhai

IPC: G06F3/01 ,  G06F1/16 ,  G06F3/0346 ,  G06F3/04817 ,  G06F3/0482 ,  G06F3/04883 ,  G06F3/04886 ,  G06F3/16

CPC classification number: G06F3/017 ,  G06F1/1694 ,  G06F3/0346 ,  G06F3/04817 ,  G06F3/0482 ,  G06F3/04883 ,  G06F3/04886 ,  G06F3/167 ,  G06F2200/1636

Abstract: An example method includes identifying, by a mobile computing device that includes a housing and a presence-sensitive display, and based on a first group of sensor signals provided at least by an inertial measurement unit included in one or more sensors, at least one first gesture that is performed at portions of the housing, wherein the one or more portions are separate from the display, initiating an interaction mode, outputting at least one visual or audio indicator for the interaction mode that is associated with a particular function of the mobile computing device, identifying, based on a third group of sensor signals provided by the one or more sensors, at least one second gesture that is performed at the one or more portions of the housing to confirm a user selection of the particular function, and, responsive to identifying the at least one second gesture, performing the particular function.

公开(公告)号：US11727212B2

公开(公告)日：2023-08-15

申请号：US17658227

申请日：2022-04-06

Applicant: Google LLC

Inventor： Shumin Zhai ,  Ciprian Ioan Chelba

IPC: G06F40/284 ,  G06F3/04886 ,  G06F3/023 ,  G06F3/04883 ,  H04L51/02 ,  G06F40/242 ,  G06F40/263 ,  H04L51/063

CPC classification number: G06F40/284 ,  G06F3/0237 ,  G06F3/04883 ,  G06F3/04886 ,  G06F40/242 ,  G06F40/263 ,  H04L51/02 ,  H04L51/063

Abstract: A computing device outputs for display at a presence-sensitive display, a graphical keyboard comprising a plurality of keys, receives an indication of at least one gesture to select a group of keys of the plurality of keys, and determines at least one characteristic associated with the at least one gesture to select the group of keys of the plurality of keys. The computing device modifies a spatial model based at least in part on the at least one characteristic and determines a candidate word based at least in part on data provided by the spatial model and a language model, wherein the spatial model provides data based at least in part on the indication of the at least one gesture and wherein the language model provides data based at least in part on a lexicon. The computing device outputs for display at the presence-sensitive display, the candidate word.

公开(公告)号：US11379663B2

公开(公告)日：2022-07-05

申请号：US17192105

申请日：2021-03-04

Applicant: Google LLC

Inventor： Yu Ouyang ,  Shumin Zhai ,  Xiaojun Bi

IPC: G06F3/04886 ,  G06F40/274 ,  G06F3/023 ,  G06F3/04842 ,  G06F3/04883 ,  G06F3/01

Abstract: A computing device outputs a keyboard for display, receives an indication of a first gesture to select a first sequence of one or more keys, determines a set of candidate strings based in part on the first sequence of keys, and outputs for display at least one of the set of candidate strings. The computing device receives an indication of a second gesture to select a second sequence of one or more keys, and determines that characters associated with the second sequence of keys are included in a first candidate word based at least in part on the set of candidate strings, or are included in a second candidate word not based on the first sequence of keys. The computing device modifies the set of candidate strings based at least in part on the determination and outputs for display at least one of the modified candidate strings.

公开(公告)号：US11150804B2

公开(公告)日：2021-10-19

申请号：US16862628

申请日：2020-04-30

Applicant: Google LLC

Inventor： Shumin Zhai ,  Thomas Breuel ,  Ouais Alsharif ,  Yu Ouyang ,  Francoise Beaufays ,  Johan Schalkwyk

IPC: G06F3/02 ,  G06F3/0488 ,  G06N3/04 ,  G06F40/232 ,  G06F40/274 ,  G06F40/279 ,  G06F3/023 ,  G06F3/0489 ,  G06F3/0482 ,  G06N3/08

Abstract: In some examples, a computing device includes at least one processor; and at least one module, operable by the at least one processor to: output, for display at an output device, a graphical keyboard; receive an indication of a gesture detected at a location of a presence-sensitive input device, wherein the location of the presence-sensitive input device corresponds to a location of the output device that outputs the graphical keyboard; determine, based on at least one spatial feature of the gesture that is processed by the computing device using a neural network, at least one character string, wherein the at least one spatial feature indicates at least one physical property of the gesture; and output, for display at the output device, based at least in part on the processing of the at least one spatial feature of the gesture using the neural network, the at least one character string.