-
公开(公告)号：US12169975B2
公开(公告)日：2024-12-17
申请号：US17596697
申请日：2020-06-17
Applicant: Snap Inc.
Inventor： Riza Alp Guler , Georgios Papandreou , Iason Kokkinos
Abstract: This specification relates to reconstructing three-dimensional (3D) scenes from two-dimensional (2D) images using a neural network. According to a first aspect of this specification, there is described a method for creating a three-dimensional reconstruction of a scene with multiple objects from a single two-dimensional image, the method comprising: receiving a single two-dimensional image; identifying all objects in the image to be reconstructed and identifying the type of said objects; estimating a three-dimensional representation of each identified object; estimating a three-dimensional plane physically supporting all three-dimensional objects; and positioning all three-dimensional objects in space relative to the supporting plane.
-
公开(公告)号：US20220375247A1
公开(公告)日：2022-11-24
申请号：US17812864
申请日：2022-07-15
Applicant: Snap inc.
Inventor： Iason Kokkinos , Georgios Papandreou , Riza Alp Guler
Abstract: Aspects of the present disclosure involve a system and a method for performing operations comprising: receiving a two-dimensional continuous surface representation of a three-dimensional object, the continuous surface comprising a plurality of landmark locations; determining a first set of soft membership functions based on a relative location of points in the two-dimensional continuous surface representation and the landmark locations; receiving a two-dimensional input image, the input image comprising an image of the object; extracting a plurality of features from the input image using a feature recognition model; generating an encoded. feature representation of the extracted features using the first set of soft membership functions; generating a dense feature representation of the extracted features from the encoded representation using a second set of soft membership functions; and processing the second set of soft membership functions and dense feature representation using a neural image decoder model to generate an output image.
-
公开(公告)号：US12254577B2
公开(公告)日：2025-03-18
申请号：US17842006
申请日：2022-06-16
Applicant: Snap Inc.
Inventor： Madiyar Aitbayev , Brian Fulkerson , Riza Alp Guler , Georgios Papandreou , Himmy Tam
Abstract: Methods and systems are disclosed for performing operations for applying augmented reality elements to a person depicted in an image. The operations include receiving an image that includes data representing a depiction of a person; extracting a portion of the image; applying a first machine learning model stage to the portion to predict a depth of a point of interest for the data representing the depiction of the person; applying a second machine learning model stage to the portion of the image to predict a relative depth of each pixel in the portion of the image to the predicted depth of the point of interest; generating dense depth reconstruction of the data representing the depiction of the person based on outputs of the first and second stages of the machine learning model; and applying one or more AR elements to the image based on the dense depth reconstruction.
-
公开(公告)号：US20230061875A1
公开(公告)日：2023-03-02
申请号：US17446533
申请日：2021-08-31
Applicant: Snap Inc.
Inventor： Yanli Zhao , Matan Zohar , Brian Fulkerson , Georgios Papandreou , Haoyang Wang
Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; obtaining an external mesh associated with an augmented reality element; automatically establishing a correspondence between the 3D body mesh associated with the real-world object and the external mesh; deforming the external mesh based on movement of the real-world object and the established correspondence with the 3D body mesh; and modifying the video to include a display of the augmented reality element based on the deformed external mesh.
-
公开(公告)号：US12148105B2
公开(公告)日：2024-11-19
申请号：US17841994
申请日：2022-06-16
Applicant: Snap Inc.
Inventor： Madiyar Aitbayev , Brian Fulkerson , Riza Alp Guler , Georgios Papandreou , Himmy Tam
Abstract: Methods and systems are disclosed for performing operations for applying augmented reality elements to a person depicted in an image. The operations include receiving an image that includes data representing a depiction of a person; generating a segmentation of the data representing the person depicted in the image; extracting a portion of the image corresponding to the segmentation of the data representing the person depicted in the image; applying a machine learning model to the portion of the image to predict a surface normal tensor for the data representing the depiction of the person, the surface normal tensor representing surface normals of each pixel within the portion of the image; and applying one or more augmented reality (AR) elements to the image based on the surface normal tensor.
-
Patent Agency Ranking
公开(公告)号：US20230267687A1
公开(公告)日：2023-08-24
申请号：US18142190
申请日：2023-05-02
Applicant: Snap Inc.
Inventor： Georgios Papandreou , Iason Kokkinos
CPC classification number: G06T17/20 , G06N20/00 , G06T2210/56 , G06T2210/32 , G06T2219/2016
Abstract: Systems and methods for reconstructing 3D models of human bodies from 2D images that counts for perspective and/or distortion effects are provided. The systems and methods include reconstructing a three-dimensional model of an object in a three-dimensional scene from a two-dimensional image comprising an image of the object. The systems and methods include determining an absolute depth of a key point of the object in the image; determining, using the absolute depth of the key point, a three-dimensional position of the key point in the three-dimensional scene; generating, using a neural network, a three-dimensional representation of the object, the three-dimensional representation comprising mesh nodes defined in a coordinate system relative to the key point; and positioning the three-dimensional representation of the object in the scene based on the position of the key point by applying a position dependent rotation to the three-dimensional object.
-
公开(公告)号：US11430247B2
公开(公告)日：2022-08-30
申请号：US16949773
申请日：2020-11-13
Applicant: Snap Inc.
Inventor： Iason Kokkinos , Georgios Papandreou , Riza Alp Guler
Abstract: Aspects of the present disclosure involve a system and a method for performing operations comprising: receiving a two-dimensional continuous surface representation of a three-dimensional object, the continuous surface comprising a plurality of landmark locations; determining a first set of soft membership functions based on a relative location of points in the two-dimensional continuous surface representation and the landmark locations; receiving a two-dimensional input image, the input image comprising an image of the object; extracting a plurality of features from the input image using a feature recognition model; generating an encoded feature representation of the extracted features using the first set of soft membership functions; generating a dense feature representation of the extracted features from the encoded representation using a second set of soft membership functions; and processing the second set of soft membership functions and dense feature representation using a neural image decoder model to generate an output image.
-
公开(公告)号：US20250061730A1
公开(公告)日：2025-02-20
申请号：US18936477
申请日：2024-11-04
Applicant: Snap Inc.
Inventor： Riza Alp Guler , Georgios Papandreou , Iason Kokkinos
Abstract: This specification relates to reconstructing three-dimensional (3D) scenes from two-dimensional (2D) images using a neural network. According to a first aspect of this specification, there is described a method for creating a three-dimensional reconstruction of a scene with multiple objects from a single two-dimensional image, the method comprising: receiving a single two-dimensional image; identifying all objects in the image to be reconstructed and identifying the type of said objects; estimating a three-dimensional representation of each identified object; estimating a three-dimensional plane physically supporting all three-dimensional objects; and positioning all three-dimensional objects in space relative to the supporting plane.
-
公开(公告)号：US12094066B2
公开(公告)日：2024-09-17
申请号：US17841994
申请日：2022-06-16
Applicant: Snap Inc.
Inventor： Madiyar Aitbayev , Brian Fulkerson , Riza Alp Guler , Georgios Papandreou , Himmy Tam
CPC classification number: G06T19/006 , G06T7/11 , G06T7/194 , G06T7/70 , G06T15/506 , G06T15/60 , G06T19/20 , G06T2200/08 , G06T2207/10016 , G06T2207/20021 , G06T2207/20081 , G06T2207/20084 , G06T2207/30196 , G06T2210/16 , G06T2219/2012
Abstract: Methods and systems are disclosed for performing operations for applying augmented reality elements to a person depicted in an image. The operations include receiving an image that includes data representing a depiction of a person; generating a segmentation of the data representing the person depicted in the image; extracting a portion of the image corresponding to the segmentation of the data representing the person depicted in the image; applying a machine learning model to the portion of the image to predict a surface normal tensor for the data representing the depiction of the person, the surface normal tensor representing surface normals of each pixel within the portion of the image; and applying one or more augmented reality (AR) elements to the image based on the surface normal tensor.
-
公开(公告)号：US20230316666A1
公开(公告)日：2023-10-05
申请号：US17842006
申请日：2022-06-16
Applicant: Snap Inc.
Inventor： Madiyar Aitbayev , Brian Fulkerson , Riza Alp Guler , Georgios Papandreou , Himmy Tam
CPC classification number: G06T19/006 , G06V10/70 , G06V10/26 , G06V20/20 , G06N20/20
Abstract: Methods and systems are disclosed for performing operations for applying augmented reality elements to a person depicted in an image. The operations include receiving an image that includes data representing a depiction of a person; extracting a portion of the image; applying a first machine learning model stage to the portion to predict a depth of a point of interest for the data representing the depiction of the person; applying a second machine learning model stage to the portion of the image to predict a relative depth of each pixel in the portion of the image to the predicted depth of the point of interest; generating dense depth reconstruction of the data representing the depiction of the person based on outputs of the first and second stages of the machine learning model; and applying one or more AR elements to the image based on the dense depth reconstruction.
-
-
-
-
-
-
-
-
-
Search Results
16
records
(took 0.022 seconds)
