公开(公告)号：US11468636B2

公开(公告)日：2022-10-11

申请号：US17222176

申请日：2021-04-05

Applicant: Snap Inc.

Inventor： Liuhao Ge ,  Zhou Ren ,  Yuncheng Li ,  Zehao Xue ,  Yingying Wang

IPC: G06T17/20 ,  G06T7/73 ,  G06N3/08 ,  G06N3/04

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for receiving a monocular image that includes a depiction of a hand and extracting features of the monocular image using a plurality of machine learning techniques. The program and method further include modeling, based on the extracted features, a pose of the hand depicted in the monocular image by adjusting skeletal joint positions of a three-dimensional (3D) hand mesh using a trained graph convolutional neural network (CNN); modeling, based on the extracted features, a shape of the hand in the monocular image by adjusting blend shape values of the 3D hand mesh representing surface features of the hand depicted in the monocular image using the trained graph CNN; and generating, for display, the 3D hand mesh adjusted to model the pose and shape of the hand depicted in the monocular image.

公开(公告)号：US10796482B2

公开(公告)日：2020-10-06

申请号：US16210927

申请日：2018-12-05

Applicant: Snap Inc.

Inventor： Liuhao Ge ,  Zhou Ren ,  Yuncheng Li ,  Zehao Xue ,  Yingying Wang

IPC: G06T17/20 ,  G06T7/73 ,  G06N3/08 ,  G06N3/04

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for receiving a monocular image that includes a depiction of a hand and extracting features of the monocular image using a plurality of machine learning techniques. The program and method further include modeling, based on the extracted features, a pose of the hand depicted in the monocular image by adjusting skeletal joint positions of a three-dimensional (3D) hand mesh using a trained graph convolutional neural network (CNN); modeling, based on the extracted features, a shape of the hand in the monocular image by adjusting blend shape values of the 3D hand mesh representing surface features of the hand depicted in the monocular image using the trained graph CNN; and generating, for display, the 3D hand mesh adjusted to model the pose and shape of the hand depicted in the monocular image.

公开(公告)号：US20220414985A1

公开(公告)日：2022-12-29

申请号：US17823764

申请日：2022-08-31

Applicant: Snap Inc.

Inventor： Liuhao Ge ,  Zhou Ren ,  Yuncheng Li ,  Zehao Xue ,  Yingying Wang

IPC: G06T17/20 ,  G06T7/73 ,  G06N3/08 ,  G06N3/04

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for receiving a monocular image that includes a depiction of a hand and extracting features of the monocular image using a plurality of machine learning techniques. The program and method further include modeling, based on the extracted features, a pose of the hand depicted in the monocular image by adjusting skeletal joint positions of a three-dimensional (3D) hand mesh using a trained graph convolutional neural network (CNN); modeling, based on the extracted features, a shape of the hand in the monocular image by adjusting blend shape values of the 3D hand mesh representing surface features of the hand depicted in the monocular image using the trained graph CNN; and generating, for display, the 3D hand mesh adjusted to model the pose and shape of the hand depicted in the monocular image.

公开(公告)号：US20210225077A1

公开(公告)日：2021-07-22

申请号：US17222176

申请日：2021-04-05

Applicant: Snap Inc.

Inventor： Liuhao Ge ,  Zhou Ren ,  Yuncheng LI ,  Zehao Xue ,  Yingying Wang

IPC: G06T17/20 ,  G06T7/73 ,  G06N3/08 ,  G06N3/04

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for receiving a monocular image that includes a depiction of a hand and extracting features of the monocular image using a plurality of machine learning techniques. The program and method further include modeling, based on the extracted features, a pose of the hand depicted in the monocular image by adjusting skeletal joint positions of a three-dimensional (3D) hand mesh using a trained graph convolutional neural network (CNN); modeling, based on the extracted features, a shape of the hand in the monocular image by adjusting blend shape values of the 3D hand mesh representing surface features of the hand depicted in the monocular image using the trained graph CNN; and generating, for display, the 3D hand mesh adjusted to model the pose and shape of the hand depicted in the monocular image.

公开(公告)号：US11734844B2

公开(公告)日：2023-08-22

申请号：US17823764

申请日：2022-08-31

Applicant: Snap Inc.

Inventor： Liuhao Ge ,  Zhou Ren ,  Yuncheng Li ,  Zehao Xue ,  Yingying Wang

IPC: G06T17/20 ,  G06T7/50 ,  G06T7/73 ,  G06N3/08 ,  G06N3/045 ,  G06V10/774 ,  G06V10/82 ,  G06V40/10

CPC classification number: G06T7/50 ,  G06N3/045 ,  G06N3/08 ,  G06T7/75 ,  G06T17/20 ,  G06V10/774 ,  G06V10/82 ,  G06V40/107 ,  G06T2207/10004

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for receiving a monocular image that includes a depiction of a hand and extracting features of the monocular image using a plurality of machine learning techniques. The program and method further include modeling, based on the extracted features, a pose of the hand depicted in the monocular image by adjusting skeletal joint positions of a three-dimensional (3D) hand mesh using a trained graph convolutional neural network (CNN); modeling, based on the extracted features, a shape of the hand in the monocular image by adjusting blend shape values of the 3D hand mesh representing surface features of the hand depicted in the monocular image using the trained graph CNN; and generating, for display, the 3D hand mesh adjusted to model the pose and shape of the hand depicted in the monocular image.

公开(公告)号：US10997787B2

公开(公告)日：2021-05-04

申请号：US17010256

申请日：2020-09-02

Applicant: Snap Inc.

Inventor： Liuhao Ge ,  Zhou Ren ,  Yuncheng Li ,  Zehao Xue ,  Yingying Wang

IPC: G06T17/20 ,  G06T7/73 ,  G06N3/04 ,  G06N3/08

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for receiving a monocular image that includes a depiction of a hand and extracting features of the monocular image using a plurality of machine learning techniques. The program and method further include modeling, based on the extracted features, a pose of the hand depicted in the monocular image by adjusting skeletal joint positions of a three-dimensional (3D) hand mesh using a trained graph convolutional neural network (CNN); modeling, based on the extracted features, a shape of the hand in the monocular image by adjusting blend shape values of the 3D hand mesh representing surface features of the hand depicted in the monocular image using the trained graph CNN; and generating, for display, the 3D hand mesh adjusted to model the pose and shape of the hand depicted in the monocular image.

公开(公告)号：US20200184721A1

公开(公告)日：2020-06-11

申请号：US16210927

申请日：2018-12-05

Applicant: Snap Inc.

Inventor： Liuhao Ge ,  Zhou Ren ,  Yuncheng Li ,  Zehao Xue ,  Yingying Wang

IPC: G06T17/20 ,  G06T7/73 ,  G06N3/04 ,  G06N3/08

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for receiving a monocular image that includes a depiction of a hand and extracting features of the monocular image using a plurality of machine learning techniques. The program and method further include modeling, based on the extracted features, a pose of the hand depicted in the monocular image by adjusting skeletal joint positions of a three-dimensional (3D) hand mesh using a trained graph convolutional neural network (CNN); modeling, based on the extracted features, a shape of the hand in the monocular image by adjusting blend shape values of the 3D hand mesh representing surface features of the hand depicted in the monocular image using the trained graph CNN; and generating, for display, the 3D hand mesh adjusted to model the pose and shape of the hand depicted in the monocular image.