-
公开(公告)号：US20250095308A1
公开(公告)日：2025-03-20
申请号：US18969689
申请日：2024-12-05
Applicant: Snap Inc.
Inventor： Matan Zohar , Yanli Zhao , Brian Fulkerson , Riza Alp Guler
Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; determining UV positions of the real-world object depicted in the video to obtain pixel values associated with the UV positions; generating an external mesh and associated augmented reality (AR) element representing the real-world object based on the pixel values associated with the UV positions; deforming the external mesh based on changes to the 3D body mesh and a deformation parameter; and modifying the video to replace the real-world object with the AR element based on the deformed external mesh.
-
公开(公告)号：US12169975B2
公开(公告)日：2024-12-17
申请号：US17596697
申请日：2020-06-17
Applicant: Snap Inc.
Inventor： Riza Alp Guler , Georgios Papandreou , Iason Kokkinos
Abstract: This specification relates to reconstructing three-dimensional (3D) scenes from two-dimensional (2D) images using a neural network. According to a first aspect of this specification, there is described a method for creating a three-dimensional reconstruction of a scene with multiple objects from a single two-dimensional image, the method comprising: receiving a single two-dimensional image; identifying all objects in the image to be reconstructed and identifying the type of said objects; estimating a three-dimensional representation of each identified object; estimating a three-dimensional plane physically supporting all three-dimensional objects; and positioning all three-dimensional objects in space relative to the supporting plane.
-
公开(公告)号：US20240303926A1
公开(公告)日：2024-09-12
申请号：US18179717
申请日：2023-03-07
Applicant: Snap Inc.
Inventor： Riza Alp Guler , Dominik Kulon , Himmy Tam , Haoyang Wang
CPC classification number: G06T17/20 , G06T7/40 , G06T11/00 , G06T2200/24 , G06T2207/10028 , G06T2207/20081 , G06T2207/30196 , G06T2210/22 , G06T2210/56
Abstract: An system for augmenting images using hand surface normal estimation is provided. In a model training phase, 3D models of hands are generated using 3D data of hands in a variety of positions. Target normal training data is generated that includes normals of surfaces of the 3D models and synthetic 2D image training data corresponding to the 3D models and the normals. The target normal training data and the synthetic image training data are used to train a normal estimation model. The normal estimation is used by an interactive application to generate augmentations that are applied to hand image data.
-
公开(公告)号：US20240013463A1
公开(公告)日：2024-01-11
申请号：US17888786
申请日：2022-08-16
Applicant: Snap Inc.
Inventor： Avihay Assouline , Itamar Berger , Riza Alp Guler , Antonios Kakolyris , Frank Lu , Haoyang Wang , Matan Zohar
CPC classification number: G06T13/40 , G06T19/006 , H04L51/10
Abstract: Aspects of the present disclosure involve a system for providing virtual experiences. The system accesses, by a messaging application, an image depicting a person. The system generates, by the messaging application, a three-dimensional (3D) avatar based on the person depicted in the image. The system receives input that selects a pose for the 3D avatar and one or more fashion items to be worn by the 3D avatar and places, by the messaging application, the 3D avatar in the selected pose and wearing the one or more fashion items in an augmented reality (AR) experience.
-
公开(公告)号：US11836866B2
公开(公告)日：2023-12-05
申请号：US17448158
申请日：2021-09-20
Applicant: Snap Inc.
Inventor： Matan Zohar , Yanli Zhao , Brian Fulkerson , Riza Alp Guler
CPC classification number: G06T19/006 , G06T7/20 , G06T7/60 , G06T11/001 , G06T17/20 , G06T2219/2016 , G06T2219/2021
Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; determining UV positions of the real-world object depicted in the video to obtain pixel values associated with the UV positions; generating an external mesh and associated augmented reality (AR) element representing the real-world object based on the pixel values associated with the UV positions; deforming the external mesh based on changes to the 3D body mesh and a deformation parameter; and modifying the video to replace the real-world object with the AR element based on the deformed external mesh.
-
Patent Agency Ranking
公开(公告)号：US20220375247A1
公开(公告)日：2022-11-24
申请号：US17812864
申请日：2022-07-15
Applicant: Snap inc.
Inventor： Iason Kokkinos , Georgios Papandreou , Riza Alp Guler
Abstract: Aspects of the present disclosure involve a system and a method for performing operations comprising: receiving a two-dimensional continuous surface representation of a three-dimensional object, the continuous surface comprising a plurality of landmark locations; determining a first set of soft membership functions based on a relative location of points in the two-dimensional continuous surface representation and the landmark locations; receiving a two-dimensional input image, the input image comprising an image of the object; extracting a plurality of features from the input image using a feature recognition model; generating an encoded. feature representation of the extracted features using the first set of soft membership functions; generating a dense feature representation of the extracted features from the encoded representation using a second set of soft membership functions; and processing the second set of soft membership functions and dense feature representation using a neural image decoder model to generate an output image.
-
公开(公告)号：US20250061730A1
公开(公告)日：2025-02-20
申请号：US18936477
申请日：2024-11-04
Applicant: Snap Inc.
Inventor： Riza Alp Guler , Georgios Papandreou , Iason Kokkinos
Abstract: This specification relates to reconstructing three-dimensional (3D) scenes from two-dimensional (2D) images using a neural network. According to a first aspect of this specification, there is described a method for creating a three-dimensional reconstruction of a scene with multiple objects from a single two-dimensional image, the method comprising: receiving a single two-dimensional image; identifying all objects in the image to be reconstructed and identifying the type of said objects; estimating a three-dimensional representation of each identified object; estimating a three-dimensional plane physically supporting all three-dimensional objects; and positioning all three-dimensional objects in space relative to the supporting plane.
-
公开(公告)号：US12198281B2
公开(公告)日：2025-01-14
申请号：US18382898
申请日：2023-10-23
Applicant: Snap Inc.
Inventor： Matan Zohar , Yanli Zhao , Brian Fulkerson , Riza Alp Guler
Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; determining UV positions of the real-world object depicted in the video to obtain pixel values associated with the UV positions; generating an external mesh and associated augmented reality (AR) element representing the real-world object based on the pixel values associated with the UV positions; deforming the external mesh based on changes to the 3D body mesh and a deformation parameter; and modifying the video to replace the real-world object with the AR element based on the deformed external mesh.
-
公开(公告)号：US12094066B2
公开(公告)日：2024-09-17
申请号：US17841994
申请日：2022-06-16
Applicant: Snap Inc.
Inventor： Madiyar Aitbayev , Brian Fulkerson , Riza Alp Guler , Georgios Papandreou , Himmy Tam
CPC classification number: G06T19/006 , G06T7/11 , G06T7/194 , G06T7/70 , G06T15/506 , G06T15/60 , G06T19/20 , G06T2200/08 , G06T2207/10016 , G06T2207/20021 , G06T2207/20081 , G06T2207/20084 , G06T2207/30196 , G06T2210/16 , G06T2219/2012
Abstract: Methods and systems are disclosed for performing operations for applying augmented reality elements to a person depicted in an image. The operations include receiving an image that includes data representing a depiction of a person; generating a segmentation of the data representing the person depicted in the image; extracting a portion of the image corresponding to the segmentation of the data representing the person depicted in the image; applying a machine learning model to the portion of the image to predict a surface normal tensor for the data representing the depiction of the person, the surface normal tensor representing surface normals of each pixel within the portion of the image; and applying one or more augmented reality (AR) elements to the image based on the surface normal tensor.
-
公开(公告)号：US20240303843A1
公开(公告)日：2024-09-12
申请号：US18179784
申请日：2023-03-07
Applicant: Snap Inc.
Inventor： Riza Alp Guler , Dominik Kulon , Himmy Tam , Haoyang Wang
CPC classification number: G06T7/55 , G06V10/25 , G06T2207/10028 , G06T2207/20081 , G06T2207/20132
Abstract: A system for generating extended reality effects using image data of hands and a depth estimation model. The depth estimation model is trained using pairings of synthetic 2D image data with sets of depths and segmentation masks. An extended reality system captures image data of hands in a real-world scene and uses the image data and the depth estimation model to generate the extended reality effects. The extended reality effects are provided to a user during an extended reality experience.
-
-
-
-
-
-
-
-
-
Search Results
26
records
(took 0.037 seconds)
