公开(公告)号：US20240303926A1

公开(公告)日：2024-09-12

申请号：US18179717

申请日：2023-03-07

Applicant: Snap Inc.

Inventor： Riza Alp Guler ,  Dominik Kulon ,  Himmy Tam ,  Haoyang Wang

IPC: G06T17/20 ,  G06T7/40 ,  G06T11/00

CPC classification number: G06T17/20 ,  G06T7/40 ,  G06T11/00 ,  G06T2200/24 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/30196 ,  G06T2210/22 ,  G06T2210/56

Abstract: An system for augmenting images using hand surface normal estimation is provided. In a model training phase, 3D models of hands are generated using 3D data of hands in a variety of positions. Target normal training data is generated that includes normals of surfaces of the 3D models and synthetic 2D image training data corresponding to the 3D models and the normals. The target normal training data and the synthetic image training data are used to train a normal estimation model. The normal estimation is used by an interactive application to generate augmentations that are applied to hand image data.

公开(公告)号：US20240303843A1

公开(公告)日：2024-09-12

申请号：US18179784

申请日：2023-03-07

Applicant: Snap Inc.

Inventor： Riza Alp Guler ,  Dominik Kulon ,  Himmy Tam ,  Haoyang Wang

IPC: G06T7/55 ,  G06V10/25

CPC classification number: G06T7/55 ,  G06V10/25 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20132

Abstract: A system for generating extended reality effects using image data of hands and a depth estimation model. The depth estimation model is trained using pairings of synthetic 2D image data with sets of depths and segmentation masks. An extended reality system captures image data of hands in a real-world scene and uses the image data and the depth estimation model to generate the extended reality effects. The extended reality effects are provided to a user during an extended reality experience.

公开(公告)号：US20230070008A1

公开(公告)日：2023-03-09

申请号：US17760424

申请日：2020-02-17

Applicant: Snap Inc.

Inventor： Dominik Kulon ,  Riza Alp Guler ,  lason Kokkinos ,  Stefanos Zafeiriou

IPC: G06T17/20 ,  G06T7/00 ,  G06T3/40

Abstract: This specification discloses methods and systems for generating three-dimensional models of deformable objects from two-dimensional images. According to one aspect of this disclosure, there is described a computer implemented method for generating a three dimensional model of deformable object from a two-dimensional image. The method comprises: receiving, as input to an embedding neural network, the two-dimensional image, wherein the two dimensional image comprises an image of an object; generating, using the embedding neural network, an embedded representation of a two-dimensional image; inputting the embedded representation into a learned decoder model; and generating, using the learned decoder model, parameters of the three dimensional model of the object from the embedded representation.