公开(公告)号：US11825065B2

公开(公告)日：2023-11-21

申请号：US17950761

申请日：2022-09-22

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Dhritiman Sagar ,  Wentao Shang

IPC: H04N13/128 ,  G06T19/00 ,  H04N13/239 ,  G06T7/593 ,  G06T7/571 ,  H04N13/111 ,  H04L67/131 ,  H04N13/00

CPC classification number: H04N13/128 ,  G06T7/571 ,  G06T7/593 ,  G06T19/006 ,  H04L67/131 ,  H04N13/111 ,  H04N13/239 ,  G06T2200/24 ,  G06T2207/30201 ,  H04N2013/0081

Abstract: The subject technology receives, at a client device, a selection of a selectable graphical item from a plurality of selectable graphical items, the selectable graphical item comprising an augmented reality content generator including a 3D effect. The subject technology applies, to image data and depth data, the 3D effect based at least in part on the augmented reality content generator, the applying the 3D effect. The subject technology generates a depth map using at least the depth data, generates a segmentation mask based at least on the image data, and performs background inpainting and blurring of the image data using at least the segmentation mask to generate background inpainted image data. The subject technology generates a 3D message based at least in part on the applied 3D effect.

公开(公告)号：US11783448B2

公开(公告)日：2023-10-10

申请号：US17328689

申请日：2021-05-24

Applicant: Snap Inc.

Inventor： Shah Tanmay Anilkumar ,  Samuel Edward Hare ,  Guohui Wang

IPC: G06T3/40 ,  G06T5/50 ,  G06T7/11 ,  G06T7/174

CPC classification number: G06T3/4076 ,  G06T5/50 ,  G06T7/11 ,  G06T7/174 ,  G06T2207/10016 ,  G06T2207/20016 ,  G06T2207/20224

Abstract: Systems and methods are provided for initiating transfer of image data corresponding to at least one predetermined level of an image pyramid comprising higher resolution to a graphic processing unit (GPU) of the computing device, calculating, by the central processing unit (CPU) of the computing device, optical flow of at least one predetermined coarse level of the image pyramid, transferring, by the CPU of the computing device, the calculated optical flow of the at least one predetermined coarse level of the image pyramid to the GPU, calculating, by the GPU of the computing device, the optical flow of the at least one predetermined level of the image pyramid comprising higher resolution, and outputting, by the GPU of the computing device, the optical flow of the image data.

公开(公告)号：US20230316683A1

公开(公告)日：2023-10-05

申请号：US18207013

申请日：2023-06-07

Applicant: Snap Inc

Inventor： Samuel Edward Hare ,  Andrew James McPhee ,  Tony Mathew

IPC: G06T19/00 ,  G06T7/70 ,  H04N9/87 ,  G11B27/036 ,  G06F3/04817 ,  G06F3/04845

CPC classification number: G06T19/006 ,  G06T7/70 ,  H04N9/8715 ,  G11B27/036 ,  G06F3/04817 ,  G06F3/04845 ,  G06F3/0488

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for rendering a three-dimensional virtual object in a video clip. The method and system include capturing, using a camera-enabled device, video content of a real-world scene and movement information collected by the camera-enabled device during capture of the video content. The captured video and movement information are stored. The stored captured video content is processed to identify a real-world object in the scene. An interactive augmented reality display is generated that: adds a virtual object to the stored video content to create augmented video content comprising the real-world scene and the virtual object; and adjusts, during playback of the augmented video content, an on-screen position of the virtual object within the augmented video content based at least in part on the stored movement information.

公开(公告)号：US11776233B2

公开(公告)日：2023-10-03

申请号：US17878635

申请日：2022-08-01

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Dhritiman Sagar ,  Wentao Shang

IPC: G06T19/20 ,  G06N20/00 ,  G06F3/04842 ,  G06F3/04883 ,  G06T15/50 ,  G06T7/507 ,  G06V40/16

CPC classification number: G06T19/20 ,  G06F3/04842 ,  G06F3/04883 ,  G06N20/00 ,  G06T7/507 ,  G06T15/50 ,  G06V40/171 ,  G06T2219/2012 ,  G06T2219/2024

Abstract: The subject technology receives a selection of a selectable graphical item from a plurality of selectable graphical items, the selectable graphical item comprising an augmented reality content generator for applying a 3D effect, the 3D effect including at least one beautification operation. The subject technology captures image data and depth data using a camera. The subject technology applies, to the image data and the depth data, the 3D effect including the at least one beautification operation based at least in part on the augmented reality content generator, the beautification operation being performed as part of applying the 3D effect. The subject technology generates a 3D message based at least in part on the applied 3D effect including the at least one beautification operation. The subject technology renders a view of the 3D message based at least in part on the applied 3D effect including the at least one beautification operation.

公开(公告)号：US11775165B2

公开(公告)日：2023-10-03

申请号：US17530181

申请日：2021-11-18

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Wentao Shang

IPC: G06T19/00 ,  G06T19/20 ,  G06T7/194 ,  G06K9/32 ,  G06F3/0488 ,  G06T7/73 ,  G06F3/04817 ,  G06F3/04847 ,  G06F3/04815 ,  G06V10/20 ,  G06V20/20 ,  G06V40/16

CPC classification number: G06F3/0488 ,  G06F3/04815 ,  G06F3/04817 ,  G06F3/04847 ,  G06T7/194 ,  G06T7/74 ,  G06T19/006 ,  G06T19/20 ,  G06V10/255 ,  G06V20/20 ,  G06V40/161 ,  G06T2200/24 ,  G06T2207/30201 ,  G06T2210/22

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for performing operations comprising: receiving, by a messaging application, an image generated by a camera of a user; displaying an option to crop an object depicted in the image, determining whether the object is depicted in the image; selectively enabling selection of the option to crop the object based on determining whether the object is depicted in the image; receiving input that selects the option to crop an object depicted in the image when the option is enabled; in response to receiving the input, segmenting the image to separate the object from other portions depicted in the image; extracting the object from the image; and adding the extracted object to a new image as a virtual object.

公开(公告)号：US11748957B2

公开(公告)日：2023-09-05

申请号：US17525612

申请日：2021-11-12

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Dhritiman Sagar ,  Wentao Shang

IPC: G06T19/00 ,  G06T7/194 ,  G06T7/50 ,  H04L67/131

CPC classification number: G06T19/006 ,  G06T7/194 ,  G06T7/50 ,  H04L67/131 ,  G06T2207/10028 ,  G06T2207/30201

Abstract: The subject technology receives image data and depth data. The subject technology selects an augmented reality content generator corresponding to a three-dimensional (3D) effect. The subject technology applies the 3D effect to the image data and the depth data based at least in part on the selected augmented reality content generator. The subject technology generates, using a processor, a message including information related to the applied 3D effect, the image data, and the depth data.

公开(公告)号：US11715268B2

公开(公告)日：2023-08-01

申请号：US17307354

申请日：2021-05-04

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Andrew James McPhee ,  Tony Mathew

IPC: G06T19/00 ,  G06T7/70 ,  G06F3/04817 ,  G06F3/04845 ,  G06F3/0488 ,  H04N9/87 ,  G11B27/036

CPC classification number: G06T19/006 ,  G06F3/04817 ,  G06F3/04845 ,  G06T7/70 ,  G11B27/036 ,  H04N9/8715 ,  G06F3/0488 ,  G06T2200/04 ,  G06T2200/24

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for rendering a three-dimensional virtual object in a video clip. The method and system include capturing, using a camera-enabled device, video content of a real-world scene and movement information collected by the camera-enabled device during capture of the video content. The captured video and movement information are stored. The stored captured video content is processed to identify a real-world object in the scene. An interactive augmented reality display is generated that: adds a virtual object to the stored video content to create augmented video content comprising the real-world scene and the virtual object and adjusts, during playback of the augmented video content, an on-screen position of the virtual object within the augmented video content based at least in part on the stored movement information.

公开(公告)号：US20230215117A1

公开(公告)日：2023-07-06

申请号：US18121893

申请日：2023-03-15

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Wentao Shang

IPC: G06T19/00 ,  G06T19/20 ,  G06T15/80 ,  G06F3/01 ,  G06F3/04883

CPC classification number: G06T19/006 ,  G06F3/012 ,  G06F3/04883 ,  G06T15/80 ,  G06T19/20 ,  G06T2219/2004

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for performing operations comprising: receiving, by a messaging application, a video feed from a camera of a user device that depicts a face; receiving a request to add a 3D caption to the video feed; identifying a graphical element that is associated with context of the 3D caption; and displaying the 3D caption and the identified graphical element in the video feed at a position in 3D space of the video feed proximate to the face depicted in the video feed.

公开(公告)号：US11676342B2

公开(公告)日：2023-06-13

申请号：US17952051

申请日：2022-09-23

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Dhritiman Sagar ,  Wentao Shang

IPC: G06T19/00 ,  G06F3/04842 ,  G06T7/50 ,  H04L51/42

CPC classification number: G06T19/00 ,  G06F3/04842 ,  G06T7/50 ,  H04L51/42 ,  G06T2207/20084

Abstract: The subject technology generates depth data using a machine learning model based at least in part on captured image data from at least one camera of a client device. The subject technology applies, to the captured image data and the generated depth data, a 3D effect based at least in part on an augmented reality content generator. The subject technology generates a depth map using at least the depth data. The subject technology generates a packed depth map based at least in part on the depth map, the generating the packed depth map. The subject technology converts a single channel floating point texture to a raw depth map. The subject technology generates multiple channels based at least in part on the raw depth map. The subject technology generates a segmentation mask based at least on the captured image data. The subject technology performs background inpainting and blurring of the captured image data using at least the segmentation mask to generate background inpainted image data.

公开(公告)号：US11657580B2

公开(公告)日：2023-05-23

申请号：US17505370

申请日：2021-10-19

Applicant: Snap Inc.

Inventor： Andrew James McPhee ,  Ebony James Charlton ,  Samuel Edward Hare ,  Michael John Evans ,  Jokubas Dargis ,  Ricardo Sanchez-Saez

IPC: G06T19/00 ,  G06T15/20 ,  G06T15/60 ,  G06F3/0346 ,  G06F3/01

CPC classification number: G06T19/006 ,  G06F3/011 ,  G06F3/0346 ,  G06T15/205 ,  G06T15/60

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for rendering three-dimensional virtual objects within real-world environments. Virtual rendering of a three-dimensional virtual object can be altered appropriately as a user moves around the object in the real-world through utilization of a redundant tracking system comprising multiple tracking sub-systems. Virtual object rendering can be with respect to a reference surface in a real-world three-dimensional space depicted in a camera view of a mobile computing device.