公开(公告)号：US20210027100A1

公开(公告)日：2021-01-28

申请号：US16992968

申请日：2020-08-13

Applicant: Snap Inc.

Inventor： Lidiia Bogdanovych ,  William Brendel ,  Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang

IPC: G06K9/62 ,  G06K9/74 ,  G06T7/11 ,  G06T7/194 ,  G06N3/08 ,  G06N3/04 ,  G06K9/68

Abstract: A machine learning system can generate an image mask (e.g., a pixel mask) comprising pixel assignments for pixels. The pixels can he assigned to classes, including, for example, face, clothes, body skin, or hair. The machine learning system can be implemented. using a convolutional neural network that is configured to execute efficiently on computing devices having limited resources, such as mobile phones. The pixel mask can be used to more accurately display video effects interacting with a user or subject depicted in the image.

公开(公告)号：US10834525B2

公开(公告)日：2020-11-10

申请号：US16293300

申请日：2019-03-05

Applicant: Snap Inc.

Inventor： Nicholas Richard Allen ,  Sheldon Chang ,  Maria Pavlovskaia ,  Amer Shahnawaz ,  Jianchao Yang

IPC: H04L12/58 ,  H04W4/02 ,  H04L29/04 ,  H04W4/021 ,  H04L29/08 ,  H04L12/18

Abstract: Systems and methods are provided for receiving a request to activate a new media collection, input increasing the default predetermined window of time that the new media collection is accessible, and input decreasing or increasing the default geographic boundary size for where media content for the media collection originates. The systems and methods further provide for sending the request to a server computer to activate the new media collection, receiving confirmation that the new media collection was activated and causing a plurality of content messages to be included in the new media collection and displayed in response to a request from at least one computing device to view the new media collection based on determining the request occurs within the increased predetermined window of time that the new media collection is accessible.

公开(公告)号：US20200320353A1

公开(公告)日：2020-10-08

申请号：US16946346

申请日：2020-06-17

Applicant: Snap Inc.

Inventor： Linjie Yang ,  Kevin Dechau Tang ,  Jianchao Yang ,  Jia Li

IPC: G06K9/62 ,  G06T11/60

Abstract: A dense captioning system and method is provided for analyzing an image to generate proposed bounding regions for a plurality of visual concepts within the image, generating a region feature for each proposed bounding region to generate a plurality of region features of the image, and determining a context feature for the image using a proposed bounding region that is a largest in size of the proposed bounding regions. For each region feature of the plurality of region features of the image, the dense captioning system and method further provides for analyzing the region feature to determine for the region feature a detection score that indicates a likelihood that the region feature comprises an actual object, and generating a caption for a visual concept in the image using the region feature and the context feature when a detection score is above a specified threshold value.

公开(公告)号：US10679428B1

公开(公告)日：2020-06-09

申请号：US15990318

申请日：2018-05-25

Applicant: Snap Inc.

Inventor： Travis Chen ,  Samuel Edward Hare ,  Yuncheng Li ,  Tony Mathew ,  Jonathan Solichin ,  Jianchao Yang ,  Ning Zhang

IPC: G06T19/20 ,  G06T7/20 ,  G06T19/00 ,  G06K9/00 ,  G06K9/32 ,  G06T7/73

Abstract: Systems, devices, media, and methods are presented for object detection and inserting graphical elements into an image stream in response to detecting the object. The systems and methods detect an object of interest in received frames of a video stream. The systems and methods identify a bounding box for the object of interest and estimate a three-dimensional position of the object of interest based on a scale of the object of interest. The systems and methods generate one or more graphical elements having a size based on the scale of the object of interest and a position based on the three-dimensional position estimated for the object of interest. The one or more graphical elements are generated within the video stream to form a modified video stream. The systems and methods cause presentation of the modified video stream including the object of interest and the one or more graphical elements.

公开(公告)号：US20200152238A1

公开(公告)日：2020-05-14

申请号：US16743563

申请日：2020-01-15

Applicant: Snap Inc.

Inventor： Jia Li ,  Nathan Litke ,  Jose Jesus (Joseph) Paredes ,  Rahul Bhupendra Sheth ,  Daniel Szeto ,  Ning Xu ,  Jianchao Yang

IPC: G11B27/10 ,  G11B27/031

Abstract: Systems and methods are described for determining a first media item related to an event, of a plurality of stored media items each comprising video content related to the event, that was captured in a device orientation corresponding to a first device orientation detected for the first computing device; providing, to the first computing device, the first media item to be displayed on the first computing device; in response to a detected change to a second device orientation for the first computing device, determining a second media item that was captured in a device orientation corresponding to the second device orientation detected for the first computing device; and providing, to the first computing device, the second media item to be displayed on the first computing device.

公开(公告)号：US10474900B2

公开(公告)日：2019-11-12

申请号：US15706096

申请日：2017-09-15

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang ,  Shah Tanmay Anilkumar

IPC: G06K9/00 ,  G06T7/246 ,  G06T1/20

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

公开(公告)号：US20190279046A1

公开(公告)日：2019-09-12

申请号：US16424404

申请日：2019-05-28

Applicant: Snap Inc.

Inventor： Wei Han ,  Jianchao Yang ,  Ning Zhang ,  Jia Li

IPC: G06K9/62 ,  G06K9/46 ,  G06T3/40 ,  G06K9/66 ,  G06K9/52

Abstract: Systems, devices, media, and methods are presented for identifying and categorically labeling objects within a set of images. The systems and methods receive an image depicting an object of interest, detect at least a portion of the object of interest within the image using a multilayer object model, determine context information, and identify the object of interest included in two or more bounding boxes.

公开(公告)号：US10055895B2

公开(公告)日：2018-08-21

申请号：US15010847

申请日：2016-01-29

Applicant: Snap Inc.

Inventor： Jia Li ,  Linjie Luo ,  Rahul Sheth ,  Ning Xu ,  Jianchao Yang

IPC: G06T19/20 ,  G06T7/20 ,  G06T19/00 ,  H04L12/58 ,  G06T7/246

CPC classification number: G06T19/20 ,  G06T7/248 ,  G06T19/006 ,  G06T2207/20104 ,  H04L51/32

Abstract: Systems and methods for local augmented reality (AR) tracking of an AR object are disclosed. In one example embodiment a device captures a series of video image frames. A user input is received at the device associating a first portion of a first image of the video image frames with an AR sticker object and a target. A first target template is generated to track the target across frames of the video image frames. In some embodiments, global tracking based on a determination that the target is outside a boundary area is used. The global tracking comprises using a global tracking template for tracking movement in the video image frames captured following the determination that the target is outside the boundary area. When the global tracking determines that the target is within the boundary area, local tracking is resumed along with presentation of the AR sticker object on an output display of the device.

公开(公告)号：US20180121762A1

公开(公告)日：2018-05-03

申请号：US15340675

申请日：2016-11-01

Applicant: SNAP INC.

Inventor： Wei Han ,  Jianchao Yang ,  Ning Zhang ,  Jia Li

IPC: G06K9/62 ,  G06K9/66 ,  G06K9/46 ,  G06K9/52 ,  G06T3/40

CPC classification number: G06K9/6267 ,  G06K9/4604 ,  G06K9/4628 ,  G06K9/4671 ,  G06K9/52 ,  G06K9/6256 ,  G06K9/627 ,  G06K9/66 ,  G06N3/0454 ,  G06N3/084 ,  G06T3/40

Abstract: Systems, devices, media, and methods are presented for identifying and categorically labeling objects within a set of images. The systems and methods receive an image depicting an object of interest, detect at least a portion of the object of interest within the image using a multilayer object model, determine context information, and identify the object of interest included in two or more bounding boxes.

公开(公告)号：US12075190B2

公开(公告)日：2024-08-27

申请号：US18221702

申请日：2023-07-13

Applicant: Snap Inc.

Inventor： Lidiia Bogdanovych ,  William Brendel ,  Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang

IPC: G06T7/11 ,  G06F18/214 ,  G06F18/24 ,  G06N3/04 ,  G06N3/08 ,  G06T7/194 ,  G06V10/82 ,  G06V30/19 ,  G06V30/242 ,  H04N7/14 ,  H04N5/445 ,  H04N5/76

CPC classification number: H04N7/147 ,  G06F18/214 ,  G06F18/24765 ,  G06N3/04 ,  G06N3/08 ,  G06T7/11 ,  G06T7/194 ,  G06V10/82 ,  G06V30/19173 ,  G06V30/242 ,  G06T2207/10016 ,  G06T2207/10024 ,  G06T2207/20024 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221 ,  G06T2207/30201 ,  H04N5/44504 ,  H04N5/76 ,  H04N7/141

Abstract: A machine learning system can generate an image mask (e.g., a pixel mask) comprising pixel assignments for pixels. The pixels can be assigned to classes, including, for example, face, clothes, body skin, or hair. The machine learning system can be implemented using a convolutional neural network that is configured to execute efficiently on computing devices having limited resources, such as mobile phones. The pixel mask can be used to more accurately display video effects interacting with a user or subject depicted in the image.