公开(公告)号：US11908082B2

公开(公告)日：2024-02-20

申请号：US18110128

申请日：2023-02-15

Applicant: Snap Inc.

Inventor： Piers George Cowburn ,  Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  David Li ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Isac Andreas Müller Sandvik ,  Wentao Shang

IPC: G06T17/20 ,  G06T19/20 ,  G06T11/20

CPC classification number: G06T17/20 ,  G06T11/203 ,  G06T19/20 ,  G06T2219/2012 ,  G06T2219/2021

Abstract: Systems and methods are provided for determining a location of a selection in a space viewable in a camera view on a display of a computing device, detecting movement of the computing device, and generating a path based on the location of the selection and the movement of the computing device. The systems and methods further provide for generating a three-dimensional (3D) mesh along the path, populating the 3D mesh with selected options to generate a 3D paint object, and causing the generated 3D paint object to be displayed. The systems and methods further provide for receiving a request to send a message comprising an image or video overlaid by the 3D paint object, capturing the image or video overlaid by the displayed 3D paint object, generating the message comprising the image or video overlaid by the 3D paint object, and sending the message to another computing device.

公开(公告)号：US20240029373A1

公开(公告)日：2024-01-25

申请号：US18375693

申请日：2023-10-02

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Wentao Shang

IPC: G06T19/00 ,  G06F3/04883 ,  G06F3/04847 ,  G06T19/20 ,  H04L51/10 ,  G06F3/01

CPC classification number: G06T19/006 ,  G06F3/04883 ,  G06F3/04847 ,  G06T19/20 ,  H04L51/10 ,  G06F3/012 ,  G06T2219/2004

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for performing operations comprising: receiving, by one or more processors that implement a messaging application, a video feed from a camera of a user device; detecting, by the messaging application, a face in the video feed; in response to detecting the face in the video feed, retrieving a three-dimensional (3D) caption; modifying the video feed to include the 3D caption at a position in 3D space of the video feed proximate to the face; and displaying a modified video feed that includes the face and the 3D caption.

公开(公告)号：US11823341B2

公开(公告)日：2023-11-21

申请号：US17817491

申请日：2022-08-04

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Andrew James McPhee ,  Maxim Maximov Lazarov ,  Wentao Shang ,  Kyle Goodrich ,  Tony Mathew

IPC: G06T19/00 ,  G06T19/20 ,  H04L51/04 ,  H04N5/272 ,  H04N23/90

CPC classification number: G06T19/006 ,  G06T19/20 ,  H04L51/04 ,  H04N5/272 ,  H04N23/90 ,  G06T2200/24 ,  G06T2219/2016

Abstract: Systems and methods are provided for capturing by a camera of a user device, a first image depicting a first environment of the user device; overlaying a first virtual object on a portion of the first image depicting the first environment; modifying a surface of the first virtual object using content captured by the user device; storing a second virtual object comprising the first virtual object with the modified surface; and generating for display the second virtual object on a portion of a second image depicting a second environment.

公开(公告)号：US20230362331A1

公开(公告)日：2023-11-09

申请号：US18221702

申请日：2023-07-13

Applicant: Snap Inc.

Inventor： Lidiia Bogdanovych ,  William Brendel ,  Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang

IPC: H04N7/14 ,  G06T7/11 ,  G06T7/194 ,  G06N3/08 ,  G06N3/04 ,  G06V30/242 ,  G06V10/82 ,  G06F18/214 ,  G06F18/24 ,  G06V30/19

CPC classification number: H04N7/147 ,  G06T7/11 ,  G06T7/194 ,  G06N3/08 ,  G06N3/04 ,  G06V30/242 ,  G06V10/82 ,  G06F18/214 ,  G06F18/24765 ,  G06V30/19173 ,  H04N7/141 ,  G06T2207/10024 ,  G06T2207/10016 ,  G06T2207/20024 ,  G06T2207/20221 ,  G06T2207/30201 ,  G06T2207/20081 ,  G06T2207/20084 ,  H04N5/44504

Abstract: A machine learning system can generate an image mask (e.g., a pixel mask) comprising pixel assignments for pixels. The pixels can be assigned to classes, including, for example, face, clothes, body skin, or hair. The machine learning system can be implemented using a convolutional neural network that is configured to execute efficiently on computing devices having limited resources, such as mobile phones. The pixel mask can be used to more accurately display video effects interacting with a user or subject depicted in the image.

公开(公告)号：US11810220B2

公开(公告)日：2023-11-07

申请号：US17581093

申请日：2022-01-21

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Wentao Shang

IPC: G06T19/00 ,  G06F3/04883 ,  G06F3/04847 ,  G06T19/20 ,  H04L51/10 ,  G06F3/01

CPC classification number: G06T19/006 ,  G06F3/012 ,  G06F3/04847 ,  G06F3/04883 ,  G06T19/20 ,  H04L51/10 ,  G06T2219/2004

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for performing operations comprising: receiving, by one or more processors that implement a messaging application, a video feed from a camera of a user device; detecting, by the messaging application, a face in the video feed; in response to detecting the face in the video feed, retrieving a three-dimensional (3D) caption; modifying the video feed to include the 3D caption at a position in 3D space of the video feed proximate to the face; and displaying a modified video feed that includes the face and the 3D caption.

公开(公告)号：US11758080B2

公开(公告)日：2023-09-12

申请号：US17829907

申请日：2022-06-01

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Wentao Shang

IPC: H04N5/262 ,  H04L51/046 ,  H04N5/272 ,  H04L51/10

CPC classification number: H04N5/2621 ,  H04L51/046 ,  H04L51/10 ,  H04N5/2628 ,  H04N5/272

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for performing operations comprising: receiving, by a messaging application, an image from a camera of a user device; receiving input that selects a user-customizable effects option for activating a user-customizable effects mode; in response to receiving the input, displaying an array of a plurality of effect options together with the image proximate to the user-customizable effects option; and applying a first effect associated with a first effect option of the plurality of effect options to the image.

公开(公告)号：US20230274543A1

公开(公告)日：2023-08-31

申请号：US18312479

申请日：2023-05-04

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang ,  Shah Tanmay Anilkumar

IPC: G06V20/40 ,  G06T7/246 ,  G06T1/20

CPC classification number: G06V20/40 ,  G06T7/248 ,  G06V20/46 ,  G06T1/20 ,  G06T2207/20081 ,  G06T2200/28 ,  G06T2207/10016

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

公开(公告)号：US20230089838A1

公开(公告)日：2023-03-23

申请号：US18073280

申请日：2022-12-01

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Wentao Shang

IPC: H04L5/00 ,  H04L27/18

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for rendering three-dimensional captions (3D) in real-world environments depicted in image content. An editing interface is displayed on a client device. The editing interface includes an input component displayed with a view of a camera feed. A first input comprising one or more text characters is received. In response to receiving the first input, a two-dimensional (2D) representation of the one or more text characters is displayed. In response to detecting a second input, a preview interface is displayed. Within the preview interface, a 3D caption based on the one or more text characters is rendered at a position in a 3D space captured within the camera feed. A message is generated that includes the 3D caption rendered at the position in the 3D space captured within the camera feed.

公开(公告)号：US20230017627A1

公开(公告)日：2023-01-19

申请号：US17952051

申请日：2022-09-23

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Dhritiman Sagar ,  Wentao Shang

IPC: G06T19/00 ,  G06F3/04842 ,  G06T7/50 ,  H04L51/42

Abstract: The subject technology generates depth data using a machine learning model based at least in part on captured image data from at least one camera of a client device. The subject technology applies, to the captured image data and the generated depth data, a 3D effect based at least in part on an augmented reality content generator. The subject technology generates a depth map using at least the depth data. The subject technology generates a packed depth map based at least in part on the depth map, the generating the packed depth map. The subject technology converts a single channel floating point texture to a raw depth map. The subject technology generates multiple channels based at least in part on the raw depth map. The subject technology generates a segmentation mask based at least on the captured image data. The subject technology performs background inpainting and blurring of the captured image data using at least the segmentation mask to generate background inpainted image data.

公开(公告)号：US20230015194A1

公开(公告)日：2023-01-19

申请号：US17950761

申请日：2022-09-22

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Dhritiman Sagar ,  Wentao Shang

IPC: H04N13/128 ,  G06T7/571 ,  H04N13/111 ,  H04L67/131 ,  H04N13/239 ,  G06T19/00 ,  G06T7/593 ,  H04N13/00

Abstract: The subject technology receives, at a client device, a selection of a selectable graphical item from a plurality of selectable graphical items, the selectable graphical item comprising an augmented reality content generator including a 3D effect. The subject technology applies, to image data and depth data, the 3D effect based at least in part on the augmented reality content generator, the applying the 3D effect. The subject technology generates a depth map using at least the depth data, generates a segmentation mask based at least on the image data, and performs background inpainting and blurring of the image data using at least the segmentation mask to generate background inpainted image data. The subject technology generates a 3D message based at least in part on the applied 3D effect.