公开(公告)号：US20240249522A1

公开(公告)日：2024-07-25

申请号：US18624757

申请日：2024-04-02

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang ,  Shah Tanmay Anilkumar

IPC: G06V20/40 ,  G06T1/20 ,  G06T7/246

CPC classification number: G06V20/40 ,  G06T1/20 ,  G06T7/248 ,  G06V20/46 ,  G06T2200/28 ,  G06T2207/10016 ,  G06T2207/20081

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

公开(公告)号：US20230290174A1

公开(公告)日：2023-09-14

申请号：US18318556

申请日：2023-05-16

Applicant: Snap Inc.

Inventor： Yuncheng Li ,  Linjie Yang ,  Ning Zhang ,  Zhengyuan Yang

IPC: G06V40/10 ,  G06F17/15 ,  G06N3/04 ,  G06T7/136

CPC classification number: G06V40/107 ,  G06F17/15 ,  G06N3/04 ,  G06T7/136

Abstract: Segmentation of an image into individual body parts is performed based on a trained model. The model is trained with a plurality of training images, each training image representing a corresponding training figure. The model is also trained with a corresponding plurality of segmentations of the training figures. Each segmentation is generated by positioning body parts between defined positions of joints of the represented figure. The body parts are represented by body part templates obtained from a template library, with the templates defining characteristics of body parts represented by the templates.

公开(公告)号：US11645843B2

公开(公告)日：2023-05-09

申请号：US17248393

申请日：2021-01-22

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang ,  Shah Tanmay Anilkumar

IPC: G06V20/40 ,  G06T7/246 ,  G06T1/20

CPC classification number: G06V20/40 ,  G06T1/20 ,  G06T7/248 ,  G06V20/46 ,  G06T2200/28 ,  G06T2207/10016 ,  G06T2207/20081

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

公开(公告)号：US20230135137A1

公开(公告)日：2023-05-04

申请号：US18090577

申请日：2022-12-29

Applicant: Snap Inc.

Inventor： Linjie Yang ,  Jianchao Yang ,  Xuehan Xiong ,  Yanran Wang

IPC: G06N3/045 ,  G06N3/08 ,  G06T7/10 ,  G06F17/18 ,  H04L51/52

Abstract: A modulated segmentation system can use a modulator network to emphasize spatial prior data of an object to track the object across multiple images. The modulated segmentation system can use a segmentation network that receives spatial prior data as intermediate data that improves segmentation accuracy. The segmentation network can further receive visual guide information from a visual guide network to increase tracking accuracy via segmentation.

公开(公告)号：US20210216776A1

公开(公告)日：2021-07-15

申请号：US17248393

申请日：2021-01-22

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang ,  Shah Tanmay Anilkumar

IPC: G06K9/00 ,  G06T7/246 ,  G06T1/20

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

公开(公告)号：US12159215B2

公开(公告)日：2024-12-03

申请号：US18489730

申请日：2023-10-18

Applicant: Snap Inc.

Inventor： Linjie Yang ,  Jianchao Yang ,  Xuehan Xiong ,  Yanran Wang

IPC: G06T7/00 ,  G06F17/18 ,  G06N3/045 ,  G06N3/08 ,  G06T7/10 ,  H04L51/52

Abstract: A modulated segmentation system can use a modulator network to emphasize spatial prior data of an object to track the object across multiple images. The modulated segmentation system can use a segmentation network that receives spatial prior data as intermediate data that improves segmentation accuracy. The segmentation network can further receive visual guide information from a visual guide network to increase tracking accuracy via segmentation.

公开(公告)号：US20230362331A1

公开(公告)日：2023-11-09

申请号：US18221702

申请日：2023-07-13

Applicant: Snap Inc.

Inventor： Lidiia Bogdanovych ,  William Brendel ,  Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang

IPC: H04N7/14 ,  G06T7/11 ,  G06T7/194 ,  G06N3/08 ,  G06N3/04 ,  G06V30/242 ,  G06V10/82 ,  G06F18/214 ,  G06F18/24 ,  G06V30/19

CPC classification number: H04N7/147 ,  G06T7/11 ,  G06T7/194 ,  G06N3/08 ,  G06N3/04 ,  G06V30/242 ,  G06V10/82 ,  G06F18/214 ,  G06F18/24765 ,  G06V30/19173 ,  H04N7/141 ,  G06T2207/10024 ,  G06T2207/10016 ,  G06T2207/20024 ,  G06T2207/20221 ,  G06T2207/30201 ,  G06T2207/20081 ,  G06T2207/20084 ,  H04N5/44504

Abstract: A machine learning system can generate an image mask (e.g., a pixel mask) comprising pixel assignments for pixels. The pixels can be assigned to classes, including, for example, face, clothes, body skin, or hair. The machine learning system can be implemented using a convolutional neural network that is configured to execute efficiently on computing devices having limited resources, such as mobile phones. The pixel mask can be used to more accurately display video effects interacting with a user or subject depicted in the image.

公开(公告)号：US20230274543A1

公开(公告)日：2023-08-31

申请号：US18312479

申请日：2023-05-04

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang ,  Shah Tanmay Anilkumar

IPC: G06V20/40 ,  G06T7/246 ,  G06T1/20

CPC classification number: G06V20/40 ,  G06T7/248 ,  G06V20/46 ,  G06T1/20 ,  G06T2207/20081 ,  G06T2200/28 ,  G06T2207/10016

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

公开(公告)号：US10929673B2

公开(公告)日：2021-02-23

申请号：US16654898

申请日：2019-10-16

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang ,  Shah Tanmay Anilkumar

IPC: G06K9/00 ,  G06T7/246 ,  G06T1/20

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

公开(公告)号：US20210027100A1

公开(公告)日：2021-01-28

申请号：US16992968

申请日：2020-08-13

Applicant: Snap Inc.

Inventor： Lidiia Bogdanovych ,  William Brendel ,  Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang

IPC: G06K9/62 ,  G06K9/74 ,  G06T7/11 ,  G06T7/194 ,  G06N3/08 ,  G06N3/04 ,  G06K9/68

Abstract: A machine learning system can generate an image mask (e.g., a pixel mask) comprising pixel assignments for pixels. The pixels can he assigned to classes, including, for example, face, clothes, body skin, or hair. The machine learning system can be implemented. using a convolutional neural network that is configured to execute efficiently on computing devices having limited resources, such as mobile phones. The pixel mask can be used to more accurately display video effects interacting with a user or subject depicted in the image.