-
公开(公告)号:US20240087266A1
公开(公告)日:2024-03-14
申请号:US17973295
申请日:2022-10-25
Applicant: Snap Inc.
Inventor: Riza Alp Guler , Himmy Tam , Haoyang Wang , Antonios Kakolyris
CPC classification number: G06T19/20 , G06T3/0093 , G06T7/10 , G06T19/006 , G06T2200/04 , G06T2207/10016 , G06T2207/20021 , G06T2207/20081 , G06T2207/20084 , G06T2207/30196 , G06T2210/22 , G06T2219/2021
Abstract: Methods and systems are disclosed for performing real-time deforming operations. The system receives an image that includes a depiction of a real-world object. The system applies a machine learning model to the image to generate a warping field and segmentation mask, the machine learning model trained to establish a relationship between a plurality of training images depicting real-world objects and corresponding ground-truth warping fields and segmentation masks associated with a target shape. The system applies the generated warping field and segmentation mask to the image to warp the real-world object depicted in the image to the target shape.
-
公开(公告)号:US11816850B2
公开(公告)日:2023-11-14
申请号:US17301926
申请日:2021-04-19
Applicant: Snap Inc.
Inventor: Riza Alp Guler , Iason Kokkinos
IPC: G06T7/73 , G06T17/00 , G06T19/20 , G06T7/50 , G06N3/02 , G06V20/64 , G06V10/764 , G06V10/82 , G06V10/44 , G06V10/22 , G06V40/10
CPC classification number: G06T7/50 , G06N3/02 , G06T7/75 , G06T17/005 , G06T19/20 , G06V10/225 , G06V10/454 , G06V10/764 , G06V10/82 , G06V20/64 , G06V40/103 , G06T2207/20084 , G06T2207/20221 , G06T2207/30196 , G06T2219/2004
Abstract: This disclosure relates to reconstructing three-dimensional models of objects from two-dimensional images. According to a first aspect, this specification describes a computer implemented method for creating a three-dimensional reconstruction from a two-dimensional image, the method comprising: receiving a two-dimensional image; identifying an object in the image to be reconstructed and identifying a type of said object; spatially anchoring a pre-determined set of object landmarks within the image; extracting a two-dimensional image representation from each object landmark; estimating a respective three-dimensional representation for the respective two-dimensional image representations; and combining the respective three-dimensional representations resulting in a fused three-dimensional representation of the object.
-
公开(公告)号:US20230316666A1
公开(公告)日:2023-10-05
申请号:US17842006
申请日:2022-06-16
Applicant: Snap Inc.
Inventor: Madiyar Aitbayev , Brian Fulkerson , Riza Alp Guler , Georgios Papandreou , Himmy Tam
CPC classification number: G06T19/006 , G06V10/70 , G06V10/26 , G06V20/20 , G06N20/20
Abstract: Methods and systems are disclosed for performing operations for applying augmented reality elements to a person depicted in an image. The operations include receiving an image that includes data representing a depiction of a person; extracting a portion of the image; applying a first machine learning model stage to the portion to predict a depth of a point of interest for the data representing the depiction of the person; applying a second machine learning model stage to the portion of the image to predict a relative depth of each pixel in the portion of the image to the predicted depth of the point of interest; generating dense depth reconstruction of the data representing the depiction of the person based on outputs of the first and second stages of the machine learning model; and applying one or more AR elements to the image based on the dense depth reconstruction.
-
公开(公告)号:US20230316665A1
公开(公告)日:2023-10-05
申请号:US17841994
申请日:2022-06-16
Applicant: Snap Inc.
Inventor: Madiyar Aitbayev , Brian Fulkerson , Riza Alp Guler , Georgios Papandreou , Himmy Tam
CPC classification number: G06T19/006 , G06T7/11 , G06T7/194 , G06T15/506 , G06T7/70 , G06T19/20 , G06T15/60 , G06T2207/30196 , G06T2207/20021 , G06T2207/20081 , G06T2207/10016 , G06T2219/2012 , G06T2207/20084 , G06T2200/08 , G06T2210/16
Abstract: Methods and systems are disclosed for performing operations for applying augmented reality elements to a person depicted in an image. The operations include receiving an image that includes data representing a depiction of a person; generating a segmentation of the data representing the person depicted in the image; extracting a portion of the image corresponding to the segmentation of the data representing the person depicted in the image; applying a machine learning model to the portion of the image to predict a surface normal tensor for the data representing the depiction of the person, the surface normal tensor representing surface normals of each pixel within the portion of the image; and applying one or more augmented reality (AR) elements to the image based on the surface normal tensor.
-
公开(公告)号:US20230090645A1
公开(公告)日:2023-03-23
申请号:US17448158
申请日:2021-09-20
Applicant: Snap Inc
Inventor: Matan Zohar , Yanli Zhao , Brian Fulkerson , Riza Alp Guler
Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; determining UV positions of the real-world object depicted in the video to obtain pixel values associated with the UV positions; generating an external mesh and associated augmented reality (AR) element representing the real-world object based on the pixel values associated with the UV positions; deforming the external mesh based on changes to the 3D body mesh and a deformation parameter; and modifying the video to replace the real-world object with the AR element based on the deformed external mesh.
-
公开(公告)号:US20230070008A1
公开(公告)日:2023-03-09
申请号:US17760424
申请日:2020-02-17
Applicant: Snap Inc.
Inventor: Dominik Kulon , Riza Alp Guler , lason Kokkinos , Stefanos Zafeiriou
Abstract: This specification discloses methods and systems for generating three-dimensional models of deformable objects from two-dimensional images. According to one aspect of this disclosure, there is described a computer implemented method for generating a three dimensional model of deformable object from a two-dimensional image. The method comprises: receiving, as input to an embedding neural network, the two-dimensional image, wherein the two dimensional image comprises an image of an object; generating, using the embedding neural network, an embedded representation of a two-dimensional image; inputting the embedded representation into a learned decoder model; and generating, using the learned decoder model, parameters of the three dimensional model of the object from the embedded representation.
-
-
-
-
-