公开(公告)号：US10679428B1

公开(公告)日：2020-06-09

申请号：US15990318

申请日：2018-05-25

Applicant: Snap Inc.

Inventor： Travis Chen ,  Samuel Edward Hare ,  Yuncheng Li ,  Tony Mathew ,  Jonathan Solichin ,  Jianchao Yang ,  Ning Zhang

IPC: G06T19/20 ,  G06T7/20 ,  G06T19/00 ,  G06K9/00 ,  G06K9/32 ,  G06T7/73

Abstract: Systems, devices, media, and methods are presented for object detection and inserting graphical elements into an image stream in response to detecting the object. The systems and methods detect an object of interest in received frames of a video stream. The systems and methods identify a bounding box for the object of interest and estimate a three-dimensional position of the object of interest based on a scale of the object of interest. The systems and methods generate one or more graphical elements having a size based on the scale of the object of interest and a position based on the three-dimensional position estimated for the object of interest. The one or more graphical elements are generated within the video stream to form a modified video stream. The systems and methods cause presentation of the modified video stream including the object of interest and the one or more graphical elements.

公开(公告)号：US10515480B1

公开(公告)日：2019-12-24

申请号：US16226108

申请日：2018-12-19

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Ebony James Charlton ,  Andrew James McPhee ,  Michael John Evans

IPC: G06T17/20 ,  G06K9/00 ,  G06T7/55

Abstract: In various example embodiments, a system and methods are presented for generation and manipulation of three dimensional (3D) models. The system and methods cause presentation of an interface frame encompassing a field of view of an image capture device. The systems and methods detect an object of interest within the interface frame, generate a movement instruction with respect to the object of interest, and detect a first change in position and a second change in position of the object of interest. The systems and methods generate a 3D model of the object of interest based on the first change in position and the second change in position.

公开(公告)号：US10474900B2

公开(公告)日：2019-11-12

申请号：US15706096

申请日：2017-09-15

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang ,  Shah Tanmay Anilkumar

IPC: G06K9/00 ,  G06T7/246 ,  G06T1/20

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

公开(公告)号：US10409569B1

公开(公告)日：2019-09-10

申请号：US15859004

申请日：2017-12-29

Applicant: Snap Inc.

Inventor： Guohui Wang ,  Fenglei Tian ,  Samuel Edward Hare ,  Sumant Hanumante ,  Tony Mathew

IPC: G06F9/44 ,  G06F8/41 ,  G06N3/04 ,  G06F9/50

Abstract: Among other things, embodiments of the present disclosure improve the functionality of computer software and systems by facilitating the automatic performance optimization of a software application based on the particular platform upon which the application runs. In some embodiments, the system can automatically choose a set of parameters or methods at run-time from a design space with pre-selected optimization methods and parameters (e.g., algorithms, software libraries, and/or hardware accelerators) for a specific task.

公开(公告)号：US10198859B2

公开(公告)日：2019-02-05

申请号：US15816795

申请日：2017-11-17

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Ebony James Charlton ,  Andrew James McPhee ,  Michael John Evans

IPC: G06K9/00 ,  G06T7/55 ,  G06T17/20

Abstract: In various example embodiments, a system and methods are presented for generation and manipulation of three dimensional (3D) models. The system and methods cause presentation of an interface frame encompassing a field of view of an image capture device. The systems and methods detect an object of interest within the interface frame, generate a movement instruction with respect to the object of interest, and detect a first change in position and a second change in position of the object of interest. The systems and methods generate a 3D model of the object of interest based on the first change in position and the second change in position.

公开(公告)号：US12211159B2

公开(公告)日：2025-01-28

申请号：US18380577

申请日：2023-10-16

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Andrew James McPhee ,  Maxim Maximov Lazarov ,  Wentao Shang ,  Kyle Goodrich ,  Tony Mathew

IPC: G06T19/00 ,  G06T19/20 ,  H04L51/04 ,  H04N5/272 ,  H04N23/90

Abstract: Systems and methods are provided for capturing by a camera of a user device, a first image depicting a first environment of the user device; overlaying a first virtual object on a portion of the first image depicting the first environment; modifying a surface of the first virtual object using content captured by the user device; storing a second virtual object comprising the first virtual object with the modified surface; and generating for display the second virtual object on a portion of a second image depicting a second environment.

公开(公告)号：US12182951B2

公开(公告)日：2024-12-31

申请号：US18240373

申请日：2023-08-31

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Wentao Shang

IPC: G06T19/00 ,  G06F3/04815 ,  G06F3/0482 ,  G06F3/0488 ,  G06T5/77

Abstract: The subject technology generates a segmentation mask based on first image data. The subject technology applies the segmentation mask on first depth data to reduce a set of artifacts in a depth map based on the first depth data. The subject technology generates a packed depth map based at least in part on the depth map. The subject technology converts a single channel floating point texture to a raw depth map. The subject technology generates multiple channels. The subject technology applies, to the first image data and the first depth data, a first augmented reality content generator corresponding to a selected first selectable graphical item, the first image data and the first depth data being captured with a camera. The subject technology generates a message including the applied first augmented reality content generator to the first image data and the first depth data.

公开(公告)号：US12094063B2

公开(公告)日：2024-09-17

申请号：US17932019

申请日：2022-09-14

Applicant: Snap Inc.

Inventor： Andrew James McPhee ,  Samuel Edward Hare ,  Peicheng Yu ,  Robert Cornelius Murphy ,  Dhritiman Sagar

IPC: G06T19/00 ,  A63F13/211 ,  G06F3/01 ,  G06F3/03 ,  G06F3/0346 ,  G06F3/038 ,  G06F11/08 ,  G06T7/246 ,  G06T15/20

CPC classification number: G06T19/003 ,  A63F13/211 ,  G06F3/011 ,  G06F3/0304 ,  G06F3/0346 ,  G06F3/038 ,  G06F11/08 ,  G06T7/246 ,  G06T15/20 ,  G06T19/006 ,  G06F2203/0381 ,  G06F2203/0382

Abstract: A redundant tracking system comprising multiple redundant tracking sub-systems, enabling seamless transitions between such tracking sub-systems, provides a solution to this problem by merging multiple tracking approaches into a single tracking system. This system is able to combine tracking objects with six degrees of freedom (6 DoF) and 3 DoF through combining and transitioning between multiple tracking systems based on the availability of tracking indicia tracked by the tracking systems. Thus, as the indicia tracked by any one tracking system becomes unavailable, the redundant tracking system seamlessly switches between tracking in 6 DoF and 3 DoF thereby providing the user with an uninterrupted experience.

公开(公告)号：US20240249522A1

公开(公告)日：2024-07-25

申请号：US18624757

申请日：2024-04-02

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang ,  Shah Tanmay Anilkumar

IPC: G06V20/40 ,  G06T1/20 ,  G06T7/246

CPC classification number: G06V20/40 ,  G06T1/20 ,  G06T7/248 ,  G06V20/46 ,  G06T2200/28 ,  G06T2207/10016 ,  G06T2207/20081

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

公开(公告)号：US11961189B2

公开(公告)日：2024-04-16

申请号：US18144058

申请日：2023-05-05

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Dhritiman Sagar ,  Wentao Shang

IPC: G06T19/00 ,  G06F3/0482 ,  G06F3/04842 ,  G06T7/50 ,  H04L51/42

CPC classification number: G06T19/00 ,  G06F3/04842 ,  G06T7/50 ,  H04L51/42 ,  G06T2207/20084

Abstract: The subject technology generates depth data using a machine learning model based at least in part on captured image data from at least one camera of a client device. The subject technology applies, to the captured image data and the generated depth data, a 3D effect based at least in part on an augmented reality content generator. The subject technology generates a depth map using at least the depth data. The subject technology generates a packed depth map based at least in part on the depth map, the generating the packed depth map. The subject technology converts a single channel floating point texture to a raw depth map. The subject technology generates multiple channels based at least in part on the raw depth map. The subject technology generates a segmentation mask based at least on the captured image data. The subject technology performs background inpainting and blurring of the captured image data using at least the segmentation mask to generate background inpainted image data.