公开(公告)号：US12217374B2

公开(公告)日：2025-02-04

申请号：US18132940

申请日：2023-04-10

Applicant: Snap Inc.

Inventor： Andrew James McPhee ,  Ebony James Charlton ,  Samuel Edward Hare ,  Michael John Evans ,  Jokubas Dargis ,  Ricardo Sanchez-Saez

IPC: G06T19/00 ,  G06F3/01 ,  G06F3/0346 ,  G06T15/20 ,  G06T15/60

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for rendering three-dimensional virtual objects within real-world environments. Virtual rendering of a three-dimensional virtual object can be altered appropriately as a user moves around the object in the real-world through utilization of a redundant tracking system comprising multiple tracking sub-systems. Virtual object rendering can be with respect to a reference surface in a real-world three-dimensional space depicted in a camera view of a mobile computing device.

公开(公告)号：US12192667B2

公开(公告)日：2025-01-07

申请号：US18224228

申请日：2023-07-20

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Wentao Shang

IPC: H04N5/262 ,  H04L51/046 ,  H04L51/10 ,  H04N5/272

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for performing operations comprising: receiving, by a messaging application, an image from a camera of a user device; receiving input that selects a user-customizable effects option for activating a user-customizable effects mode; in response to receiving the input, displaying an array of a plurality of effect options together with the image proximate to the user-customizable effects option; and applying a first effect associated with a first effect option of the plurality of effect options to the image.

公开(公告)号：US20240372963A1

公开(公告)日：2024-11-07

申请号：US18772971

申请日：2024-07-15

Applicant: Snap inc.

Inventor： Lidiia Bogdanovych ,  William Brendel ,  Samuel Edward Hare ,  Fedir Paliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang

IPC: H04N7/14 ,  G06F18/214 ,  G06F18/24 ,  G06N3/04 ,  G06N3/08 ,  G06T7/11 ,  G06T7/194 ,  G06V10/82 ,  G06V30/19 ,  G06V30/242 ,  H04N5/445 ,  H04N5/76

Abstract: A machine learning system can generate an image mask (e.g., a pixel mask) comprising pixel assignments for pixels. The pixels can be assigned to classes, including, for example, face, clothes, body skin, or hair. The machine learning system can be implemented using a convolutional neural network that is configured to execute efficiently on computing devices having limited resources, such as mobile phones. The pixel mask can be used to more accurately display video effects interacting with a user or subject depicted in the image.

公开(公告)号：US11989938B2

公开(公告)日：2024-05-21

申请号：US18312479

申请日：2023-05-04

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang ,  Shah Tanmay Anilkumar

IPC: G06V20/40 ,  G06T1/20 ,  G06T7/246

CPC classification number: G06V20/40 ,  G06T1/20 ,  G06T7/248 ,  G06V20/46 ,  G06T2200/28 ,  G06T2207/10016 ,  G06T2207/20081

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

公开(公告)号：US20240037878A1

公开(公告)日：2024-02-01

申请号：US18380577

申请日：2023-10-16

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Andrew James McPhee ,  Maxim Maximov Lazarov ,  Wentao Shang ,  Kyle Goodrich ,  Tony Mathew

IPC: G06T19/00 ,  G06T19/20 ,  H04L51/04 ,  H04N5/272 ,  H04N23/90

CPC classification number: G06T19/006 ,  G06T19/20 ,  H04L51/04 ,  H04N5/272 ,  H04N23/90 ,  G06T2200/24 ,  G06T2219/2016

Abstract: Systems and methods are provided for capturing by a camera of a user device, a first image depicting a first environment of the user device; overlaying a first virtual object on a portion of the first image depicting the first environment; modifying a surface of the first virtual object using content captured by the user device; storing a second virtual object comprising the first virtual object with the modified surface; and generating for display the second virtual object on a portion of a second image depicting a second environment.

公开(公告)号：US11694402B2

公开(公告)日：2023-07-04

申请号：US17506478

申请日：2021-10-20

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Andrew James McPhee ,  Daniel Moreno ,  Kyle Goodrich

IPC: G06T17/20 ,  G06T15/04 ,  G06T19/20 ,  G06T7/246 ,  G06T3/20 ,  G06T3/40 ,  G06T7/20 ,  G06T11/60 ,  G06T13/20 ,  G06T15/00 ,  G06T19/00

CPC classification number: G06T17/20 ,  G06T3/20 ,  G06T3/40 ,  G06T7/20 ,  G06T7/251 ,  G06T11/60 ,  G06T13/20 ,  G06T15/00 ,  G06T15/04 ,  G06T19/006 ,  G06T19/20 ,  G06F2218/00 ,  G06T2219/2004 ,  G06T2219/2012 ,  G06T2219/2016

Abstract: Systems and methods are provided for receiving a two-dimensional (2D) image comprising a 2D object; identifying a contour of the 2D object; generating a three-dimensional (3D) mesh based on the contour of the 2D object; and applying a texture of the 2D object to the 3D mesh to output a 3D object representing the 2D object.

公开(公告)号：US20230196675A1

公开(公告)日：2023-06-22

申请号：US18110128

申请日：2023-02-15

Applicant: Snap Inc.

Inventor： Piers George Cowburn ,  Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  David Li ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Isac Andreas Müller Sandvik ,  Wentao Shang

IPC: G06T17/20 ,  G06T19/20 ,  G06T11/20

CPC classification number: G06T17/20 ,  G06T11/203 ,  G06T19/20 ,  G06T2219/2012 ,  G06T2219/2021

Abstract: Systems and methods are provided for determining a location of a selection in a space viewable in a camera view on a display of a computing device, detecting movement of the computing device, and generating a path based on the location of the selection and the movement of the computing device. The systems and methods further provide for generating a three-dimensional (3D) mesh along the path, populating the 3D mesh with selected options to generate a 3D paint object, and causing the generated 3D paint object to be displayed. The systems and methods further provide for receiving a request to send a message comprising an image or video overlaid by the 3D paint object, capturing the image or video overlaid by the displayed 3D paint object, generating the message comprising the image or video overlaid by the 3D paint object, and sending the message to another computing device.

公开(公告)号：US11676381B2

公开(公告)日：2023-06-13

申请号：US17248393

申请日：2021-01-22

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang ,  Shah Tanmay Anilkumar

IPC: G06V20/40 ,  G06T7/246 ,  G06T1/20

CPC classification number: G06V20/40 ,  G06T1/20 ,  G06T7/248 ,  G06V20/46 ,  G06T2200/28 ,  G06T2207/10016 ,  G06T2207/20081

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

公开(公告)号：US11508135B2

公开(公告)日：2022-11-22

申请号：US17301735

申请日：2021-04-13

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Wentao Shang

IPC: G06T19/00 ,  G06F3/0482 ,  G06F3/0488 ,  G06T5/00 ,  G06F3/04815

Abstract: The subject technology selects a set of augmented reality content generators from available augmented reality content generators, the selected set of augmented reality content generators comprising at least one augmented reality content generator for applying a three-dimensional (3D) effect. The subject technology causes display of a carousel interface including selectable graphical items, each selectable graphical item corresponding to a respective augmented reality content generator. The subject technology receives a selection of a first selectable graphical item from the selectable graphical items, the first selectable graphical item including a first augmented reality content generator for applying a first 3D effect. The subject technology applies, to first image data and first depth data, the first augmented reality content generator corresponding to the selected first selectable graphical item. The subject technology generates a message including the applied first augmented reality content generator to the first image data and the first depth data.

公开(公告)号：US11501499B2

公开(公告)日：2022-11-15

申请号：US16723540

申请日：2019-12-20

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Wentao Shang

IPC: G06T19/00 ,  G06T7/194 ,  G06T7/11 ,  G06F3/01 ,  G06T15/20 ,  G06T19/20

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and a method for rendering virtual modifications to real-world environments depicted in image content. A reference surface is detected in a three-dimensional (3D) space captured within a camera feed produced by a camera of a computing device. An image mask is applied to the reference surface within the 3D space captured within the camera feed. A visual effect is applied to the image mask corresponding to the reference surface in the 3D space. The application of the visual effect to the image mask causes a modified surface to be rendered in presenting the camera feed on a display of the computing device.