公开(公告)号：US12164694B2

公开(公告)日：2024-12-10

申请号：US17532976

申请日：2021-11-22

Applicant: Ultrahaptics IP Two Limited

Inventor： David S. Holz ,  Raffi Bedikian ,  Adrian Gasinski ,  Maxwell Sills ,  Hua Yang ,  Gabriel Hare

IPC: G06F3/04842 ,  G06F3/01 ,  G06F3/03 ,  G06F3/042 ,  G06F3/04815 ,  G06N5/04

Abstract: The technology disclosed relates to manipulating a virtual object. In particular, it relates to detecting a hand in a three-dimensional (3D) sensory space and generating a predictive model of the hand, and using the predictive model to track motion of the hand. The predictive model includes positions of calculation points of fingers, thumb and palm of the hand. The technology disclosed relates to dynamically selecting at least one manipulation point proximate to a virtual object based on the motion tracked by the predictive model and positions of one or more of the calculation points, and manipulating the virtual object by interaction between at least some of the calculation points of the predictive model and the dynamically selected manipulation point.

公开(公告)号：US11841920B1

公开(公告)日：2023-12-12

申请号：US15432869

申请日：2017-02-14

Applicant: Ultrahaptics IP Two Limited

Inventor： Jonathan Marsden ,  Raffi Bedikian ,  David Samuel Holz

IPC: G06K9/62 ,  G06K9/00 ,  G06T7/73 ,  G06K9/78 ,  G06T7/285 ,  G06T7/246 ,  G06N3/04 ,  G06N3/08 ,  G06F3/01 ,  G06V10/10 ,  G06V20/64 ,  G06V40/20 ,  G06F18/214 ,  G06F18/24 ,  G06V10/70

CPC classification number: G06F18/214 ,  G06F18/24 ,  G06N3/04 ,  G06N3/08 ,  G06T7/248 ,  G06T7/285 ,  G06T7/74 ,  G06V10/70 ,  G06V20/64 ,  G06V40/28 ,  G06F3/011 ,  G06F3/017 ,  G06T2207/10021 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/30196

Abstract: The technology disclosed introduces two types of neural networks: “master” or “generalists” networks and “expert” or “specialists” networks. Both, master networks and expert networks, are fully connected neural networks that take a feature vector of an input hand image and produce a prediction of the hand pose. Master networks and expert networks differ from each other based on the data on which they are trained. In particular, master networks are trained on the entire data set. In contrast, expert networks are trained only on a subset of the entire dataset. In regards to the hand poses, master networks are trained on the input image data representing all available hand poses comprising the training data (including both real and simulated hand images).

公开(公告)号：US11740705B2

公开(公告)日：2023-08-29

申请号：US17666534

申请日：2022-02-07

Applicant: Ultrahaptics IP Two Limited

Inventor： Raffi Bedikian ,  Jonathan Marsden ,  Keith Mertens ,  David Holz ,  Maxwell Sills ,  Matias Perez ,  Gabriel Hare ,  Ryan Julian

IPC: G06F3/01 ,  G06F3/04815 ,  G06F3/03 ,  G06V40/20

CPC classification number: G06F3/017 ,  G06F3/0325 ,  G06F3/04815 ,  G06V40/20

Abstract: A method and system are provided for controlling a machine using gestures. The method includes sensing a variation of position of a control object using an imaging system, determining, from the variation, one or more primitives describing a characteristic of a control object moving in space, comparing the one or more primitives to one or more gesture templates in a library of gesture templates, selecting, based on a result of the comparing, one or more gesture templates corresponding to the one or more primitives, and providing at least one gesture template of the selected one or more gesture templates as an indication of a command to issue to a machine under control responsive to the variation.

公开(公告)号：US11714880B1

公开(公告)日：2023-08-01

申请号：US16508231

申请日：2019-07-10

Applicant: Ultrahaptics IP Two Limited

Inventor： Jonathan Marsden ,  Raffi Bedikian ,  David Samuel Holz

IPC: G06K9/62 ,  G06K9/00 ,  G06T7/13 ,  G06K9/46

CPC classification number: G06K9/6269 ,  G06K9/00355 ,  G06K9/4604 ,  G06T7/13 ,  G06T2207/10028

Abstract: The technology disclosed performs hand pose estimation on a so-called “joint-by-joint” basis. So, when a plurality of estimates for the 28 hand joints are received from a plurality of expert networks (and from master experts in some high-confidence scenarios), the estimates are analyzed at a joint level and a final location for each joint is calculated based on the plurality of estimates for a particular joint. This is a novel solution discovered by the technology disclosed because nothing in the field of art determines hand pose estimates at such granularity and precision. Regarding granularity and precision, because hand pose estimates are computed on a joint-by-joint basis, this allows the technology disclosed to detect in real time even the minutest and most subtle hand movements, such a bend/yaw/tilt/roll of a segment of a finger or a tilt an occluded finger, as demonstrated supra in the Experimental Results section of this application.

公开(公告)号：US11010512B2

公开(公告)日：2021-05-18

申请号：US16695136

申请日：2019-11-25

Applicant: Ultrahaptics IP Two Limited

Inventor： David S. Holz ,  Kevin Horowitz ,  Raffi Bedikian ,  Hua Yang

IPC: G06F17/10 ,  G06F30/20 ,  G06K9/00 ,  G06F3/01

Abstract: The technology disclosed relates to simplifying updating of a predictive model using clustering observed points. In particular, it relates to observing a set of points in 3D sensory space, determining surface normal directions from the points, clustering the points by their surface normal directions and adjacency, accessing a predictive model of a hand, refining positions of segments of the predictive model, matching the clusters of the points to the segments, and using the matched clusters to refine the positions of the matched segments. It also relates to distinguishing between alternative motions between two observed locations of a control object in a 3D sensory space by accessing first and second positions of a segment of a predictive model of a control object such that motion between the first position and the second position was at least partially occluded from observation in a 3D sensory space.