-
公开(公告)号：US20240070986A1
公开(公告)日：2024-02-29
申请号：US18502634
申请日：2023-11-06
Applicant: Snap Inc.
Inventor： Matan Zohar , Artem Bondich , Yanli Zhao , Brian Fulkerson
CPC classification number: G06T17/205 , G06T7/251 , G06T7/70 , G06T19/006 , G06T2207/10016
Abstract: Methods and systems are disclosed for performing operations for deforming an external mesh. The operations comprise receiving a video that includes a depiction of a real-world object. The operations comprise generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video. The operations comprise obtaining an external mesh associated with an Augmented-Reality (AR) element. The operations comprise accessing a plurality of deformation attributes associated with the external mesh, each attribute corresponding to a different deformation model. The operations comprise separately deforming, based on respective deformation models, a first portion of the external mesh and a second portion of the external mesh. The operations comprise modifying the video to include a display of the AR element based on the separately deformed first and second portions of the external mesh.
-
公开(公告)号：US11836866B2
公开(公告)日：2023-12-05
申请号：US17448158
申请日：2021-09-20
Applicant: Snap Inc.
Inventor： Matan Zohar , Yanli Zhao , Brian Fulkerson , Riza Alp Guler
CPC classification number: G06T19/006 , G06T7/20 , G06T7/60 , G06T11/001 , G06T17/20 , G06T2219/2016 , G06T2219/2021
Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; determining UV positions of the real-world object depicted in the video to obtain pixel values associated with the UV positions; generating an external mesh and associated augmented reality (AR) element representing the real-world object based on the pixel values associated with the UV positions; deforming the external mesh based on changes to the 3D body mesh and a deformation parameter; and modifying the video to replace the real-world object with the AR element based on the deformed external mesh.
-
公开(公告)号：US20230115531A1
公开(公告)日：2023-04-13
申请号：US17498394
申请日：2021-10-11
Applicant: Snap Inc.
Inventor： Matan Zohar , Artem Bondich , Yanli Zhao , Brian Fulkerson
Abstract: Methods and systems are disclosed for performing operations for deforming an external mesh. The operations comprise receiving a video that includes a depiction of a real-world object. The operations comprise generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video. The operations comprise obtaining an external mesh associated with an Augmented-Reality (AR) element. The operations comprise accessing a plurality of deformation attributes associated with the external mesh, each attribute corresponding to a different deformation model. The operations comprise separately deforming, based on respective deformation models, a first portion of the external mesh and a second portion of the external mesh. The operations comprise modifying the video to include a display of the AR element based on the separately deformed first and second portions of the external mesh.
-
公开(公告)号：US20230074826A1
公开(公告)日：2023-03-09
申请号：US17468774
申请日：2021-09-08
Applicant: Snap Inc.
Inventor： Matan Zohar , Yanli Zhao , Brian Fulkerson
Abstract: Methods and systems are disclosed for performing operations comprising: receiving video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; obtaining an external mesh associated with an augmented reality (AR) element; determining that a first portion of the external mesh is associated with movement information of the 3D body mesh; determining that a second portion of the external mesh is associated with movement information of an external force model; deforming the first and second portions of the external mesh separately based on movement information of the 3D body mesh and movement information of the external force model; and modifying the video to include a display of the AR element based on the deformed first and second portions of the external mesh.
-
公开(公告)号：US12217453B2
公开(公告)日：2025-02-04
申请号：US18231025
申请日：2023-08-07
Applicant: Snap Inc.
Inventor： Matan Zohar , Yanli Zhao , Brian Fulkerson , Itamar Berger
IPC: G06T7/70 , G06F18/214 , G06F18/24 , G06T19/00 , G06V40/10
Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for performing operations comprising: receiving a video that depicts a person. The operations further include identifying a set of skeletal joints of the person. The operations further include identifying a pose of the person depicted in the video based on positioning of the set of skeletal joints (or detecting a hand pose, detecting a mirror frame, or detecting a mobile device). The operations further include determining, based on the pose of the person (or detecting a hand pose, detecting a mirror frame, or detecting a mobile device), that the video comprises a mirror reflection of the person. The operations further include, in response to determining that the video comprises the mirror reflection of the person, causing display of a 3D virtual object in the video.
-
Patent Agency Ranking
公开(公告)号：US20240153214A1
公开(公告)日：2024-05-09
申请号：US18413423
申请日：2024-01-16
Applicant: Snap Inc.
Inventor： Yanli Zhao , Matan Zohar , Brian Fulkerson , Georgios Papandreou , Haoyang Wang
IPC: G06T17/20 , G06T19/00 , G06T19/20 , H04L51/046 , H04L51/10
CPC classification number: G06T17/205 , G06T19/006 , G06T19/20 , H04L51/046 , H04L51/10 , G06T2219/2016
Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; obtaining an external mesh associated with an augmented reality element; automatically establishing a correspondence between the 3D body mesh associated with the real-world object and the external mesh; deforming the external mesh based on movement of the real-world object and the established correspondence with the 3D body mesh; and modifying the video to include a display of the augmented reality element based on the deformed external mesh.
-
公开(公告)号：US11908083B2
公开(公告)日：2024-02-20
申请号：US17446533
申请日：2021-08-31
Applicant: Snap Inc.
Inventor： Yanli Zhao , Matan Zohar , Brian Fulkerson , Georgios Papandreou , Haoyang Wang
IPC: G06T17/20 , G06T19/00 , G06T19/20 , H04L51/046 , H04L51/10
CPC classification number: G06T17/205 , G06T19/006 , G06T19/20 , H04L51/046 , H04L51/10 , G06T2219/2016
Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; obtaining an external mesh associated with an augmented reality element; automatically establishing a correspondence between the 3D body mesh associated with the real-world object and the external mesh; deforming the external mesh based on movement of the real-world object and the established correspondence with the 3D body mesh; and modifying the video to include a display of the augmented reality element based on the deformed external mesh.
-
公开(公告)号：US20240046587A1
公开(公告)日：2024-02-08
申请号：US18382898
申请日：2023-10-23
Applicant: Snap Inc.
Inventor： Matan Zohar , Yanli Zhao , Brian Fulkerson , Riza Alp Guler
CPC classification number: G06T19/006 , G06T7/20 , G06T7/60 , G06T11/001 , G06T17/20 , G06T2219/2016 , G06T2219/2021
Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; determining UV positions of the real-world object depicted in the video to obtain pixel values associated with the UV positions; generating an external mesh and associated augmented reality (AR) element representing the real-world object based on the pixel values associated with the UV positions; deforming the external mesh based on changes to the 3D body mesh and a deformation parameter; and modifying the video to replace the real-world object with the AR element based on the deformed external mesh.
-
公开(公告)号：US20240020934A1
公开(公告)日：2024-01-18
申请号：US18475813
申请日：2023-09-27
Applicant: Snap Inc.
Inventor： Matan Zohar , Yanli Zhao , Brian Fulkerson
CPC classification number: G06T19/006 , G06T7/251 , G06T13/40 , G06T13/80 , G06T17/20 , G06T19/20 , G06T2200/24 , G06T2207/10016 , G06T2219/2021
Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; obtaining an external mesh associated with an augmented reality (AR) element; automatically establishing a correspondence between a portion of the 3D body mesh associated with the real-world object and the external mesh; blending the portion of the 3D body mesh into the external mesh according to a blending parameter; and as the portion of the 3D body mesh is being blended into the external mesh, modifying the video to depict a portion of the real-world object being changed into the AR element.
-
公开(公告)号：US20230419621A1
公开(公告)日：2023-12-28
申请号：US18243360
申请日：2023-09-07
Applicant: Snap Inc.
Inventor： Matan Zohar , Yanli Zhao , Brian Fulkerson , Itamar Berger
CPC classification number: G06T19/006 , G06T7/70 , G06V40/10 , G10L15/08 , G10L2015/088
Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for performing operations comprising receiving an image that depicts a person, identifying a set of skeletal joints of the person and identifying a pose of the person depicted in the image based on positioning of the set of skeletal joints. The operations also include receiving speech input comprising a request to perform an AR operation and an ambiguous intent, discerning the ambiguous intent of the speech input based on the pose of the person depicted in the image and in response to receiving the speech input, performing the AR operation based on discerning the ambiguous intent of the speech input based on the pose of the person depicted in the image.
-
-
-
-
-
-
-
-
-
Search Results
24
records
(took 0.022 seconds)
