公开(公告)号：US11544902B2

公开(公告)日：2023-01-03

申请号：US17319399

申请日：2021-05-13

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Wentao Shang

IPC: G06T19/20 ,  G06T17/20 ,  G06T15/04 ,  G06K9/00 ,  G06T7/246 ,  G06T3/20 ,  G06T3/40 ,  G06T7/20 ,  G06T11/60 ,  G06T13/20 ,  G06T15/00 ,  G06T19/00

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for rendering three-dimensional captions (3D) in real-world environments depicted in image content. An editing interface is displayed on a client device. The editing interface includes an input component displayed with a view of a camera feed. A first input comprising one or more text characters is received. In response to receiving the first input, a two-dimensional (2D) representation of the one or more text characters is displayed. In response to detecting a second input, a preview interface is displayed. Within the preview interface, a 3D caption based on the one or more text characters is rendered at a position in a 3D space captured within the camera feed. A message is generated that includes the 3D caption rendered at the position in the 3D space captured within the camera feed.

公开(公告)号：US20220383592A1

公开(公告)日：2022-12-01

申请号：US17819198

申请日：2022-08-11

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Ebony James Charlton ,  Andrew james McPhee ,  Michael John Evans

IPC: G06T17/20 ,  G06T7/55 ,  G06V20/64 ,  G06V40/60 ,  G06V40/16

Abstract: In various example embodiments, a system and methods are presented for generation and manipulation of three dimensional (3D) models. The system and methods cause presentation of an interface frame encompassing a field of view of an image capture device. The systems and methods detect an object of interest within the interface frame, generate a movement instruction with respect to the object of interest, and detect a first change in position and a second change in position of the object of interest. The systems and methods generate a 3D model of the object of interest based on the first change in position and the second change in position.

公开(公告)号：US11481978B2

公开(公告)日：2022-10-25

申请号：US16948381

申请日：2020-09-16

Applicant: Snap Inc.

Inventor： Andrew James McPhee ,  Samuel Edward Hare ,  Peicheng Yu ,  Robert Cornelius Murphy ,  Dhritiman Sagar

IPC: G06F3/03 ,  A63F13/211 ,  G06T19/00 ,  G06T7/246 ,  G06F3/038 ,  G06F3/0346 ,  G06F3/01 ,  G06F11/08 ,  G06T15/20

Abstract: A redundant tracking system comprising multiple redundant tracking sub-systems, enabling seamless transitions between such tracking sub-systems, provides a solution to this problem by merging multiple tracking approaches into a single tracking system. This system is able to combine tracking objects with six degrees of freedom (6DoF) and 3DoF through combining and transitioning between multiple tracking systems based on the availability of tracking indicia tracked by the tracking systems. Thus, as the indicia tracked by any one tracking system becomes unavailable, the redundant tracking system seamlessly switches between tracking in 6DoF and 3DoF thereby providing the user with an uninterrupted experience.

公开(公告)号：US11457196B2

公开(公告)日：2022-09-27

申请号：US17006507

申请日：2020-08-28

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Dhritiman Sagar ,  Wentao Shang

IPC: H04N13/128 ,  G06T19/00 ,  H04N13/239 ,  G06T7/593 ,  G06T7/571 ,  H04N13/111 ,  H04L67/131 ,  H04N13/00

Abstract: The subject technology selects a set of augmented reality content generators from a plurality of available augmented reality content generator based on metadata associated with each respective augmented reality content generator. The subject technology receives, at a client device, a selection of a selectable graphical item from a plurality of selectable graphical items, the selectable graphical item comprising an augmented reality content generator including a 3D effect. The subject technology captures image data and depth data using at least one camera of the client device. The subject technology applies, to the image data and the depth data, the 3D effect based at least in part on the augmented reality content generator.

公开(公告)号：US20220284682A1

公开(公告)日：2022-09-08

申请号：US17525612

申请日：2021-11-12

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Dhritiman Sagar ,  Wentao Shang

IPC: G06T19/00 ,  G06T7/194 ,  G06T7/50 ,  H04L67/131

Abstract: The subject technology receives image data and depth data. The subject technology selects an augmented reality content generator corresponding to a three-dimensional (3D) effect. The subject technology applies the 3D effect to the image data and the depth data based at least in part on the selected augmented reality content generator. The subject technology generates, using a processor, a message including information related to the applied 3D effect, the image data, and the depth data.

公开(公告)号：US11227442B1

公开(公告)日：2022-01-18

申请号：US16721459

申请日：2019-12-19

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Wentao Shang

IPC: G06T19/00 ,  G06T19/20 ,  G06F3/0488 ,  G06F3/01 ,  G06T15/80

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for performing operations comprising: receiving, by a messaging application, a video feed from a camera of a user device that depicts a face; receiving a request to add a 3D caption to the video feed; identifying a graphical element that is associated with context of the 3D caption; and displaying the 3D caption and the identified graphical element in the video feed at a position in 3D space of the video feed proximate to the face depicted in the video feed.

公开(公告)号：US20210287439A1

公开(公告)日：2021-09-16

申请号：US15929355

申请日：2020-04-28

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Wentao Shang

IPC: G06T19/00 ,  G06T19/20 ,  G06T7/194 ,  G06T7/73 ,  G06K9/32 ,  G06K9/46 ,  G06F3/0481 ,  G06F3/0488 ,  G06F3/0484

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for performing operations comprising: receiving, by a messaging application, an image generated by a camera of a user; displaying an option to crop an object depicted in the image; determining whether the object is depicted in the image; selectively enabling selection of the option to crop the object based on determining whether the object is depicted in the image; receiving input that selects the option to crop an object depicted in the image when the option is enabled; in response to receiving the input, segmenting the image to separate the object from other portions depicted in the image; extracting the object from the image; and adding the extracted object to a new image as a virtual object.

公开(公告)号：US10929673B2

公开(公告)日：2021-02-23

申请号：US16654898

申请日：2019-10-16

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang ,  Shah Tanmay Anilkumar

IPC: G06K9/00 ,  G06T7/246 ,  G06T1/20

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

公开(公告)号：US20210027100A1

公开(公告)日：2021-01-28

申请号：US16992968

申请日：2020-08-13

Applicant: Snap Inc.

Inventor： Lidiia Bogdanovych ,  William Brendel ,  Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang

IPC: G06K9/62 ,  G06K9/74 ,  G06T7/11 ,  G06T7/194 ,  G06N3/08 ,  G06N3/04 ,  G06K9/68

Abstract: A machine learning system can generate an image mask (e.g., a pixel mask) comprising pixel assignments for pixels. The pixels can he assigned to classes, including, for example, face, clothes, body skin, or hair. The machine learning system can be implemented. using a convolutional neural network that is configured to execute efficiently on computing devices having limited resources, such as mobile phones. The pixel mask can be used to more accurately display video effects interacting with a user or subject depicted in the image.

公开(公告)号：US20200334916A1

公开(公告)日：2020-10-22

申请号：US16922618

申请日：2020-07-07

Applicant: Snap Inc.

Inventor： Andrew James McPhee ,  Ebony James Charlton ,  Samuel Edward Hare ,  Michael John Evans ,  Jokubas Dargis ,  Ricardo Sanchez-Saez

IPC: G06T19/00 ,  G06T15/60 ,  G06F3/0346 ,  G06T15/20 ,  G06F3/01

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for rendering three-dimensional virtual objects within real-world environments. Virtual rendering of a three-dimensional virtual object can be altered appropriately as a user moves around the object in the real-world through utilization of a redundant tracking system comprising multiple tracking sub-systems. Virtual object rendering can be with respect to a reference surface in a real-world three-dimensional space depicted in a camera view of a mobile computing device.